-- Logs begin at Wed 2022-09-07 15:30:51 UTC, end at Wed 2022-09-07 16:46:32 UTC. -- Sep 07 16:02:34.595313 dv-cinder-856174-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 07 16:02:34.603184 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 7 16:02:34 2022] *** Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: nodename: dv-cinder-856174-1-iscsi Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: machine: x86_64 Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: clock source: unix Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: pcre jit disabled Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: detected number of CPU cores: 4 Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: current working directory: / Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: detected binary path: /usr/bin/uwsgi-core Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: your processes number limit is 47680 Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: your memory page size is 4096 bytes Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: detected max file descriptor number: 2048 Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: lock engine: pthread robust mutexes Sep 07 16:02:34.607097 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: thunder lock: enabled Sep 07 16:02:34.608374 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 07 16:02:34.608374 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 07 16:02:34.645947 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: Python main interpreter initialized at 0x55a49b08f9c0 Sep 07 16:02:34.645947 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: python threads support enabled Sep 07 16:02:34.645947 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: your server socket listen backlog is limited to 100 connections Sep 07 16:02:34.645947 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: your mercy for graceful operations on workers is 90 seconds Sep 07 16:02:34.646316 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: mapped 403077 bytes (393 KB) for 2 cores Sep 07 16:02:34.646393 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: *** Operational MODE: preforking *** Sep 07 16:02:34.646642 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: *** uWSGI is running in multiple interpreter mode *** Sep 07 16:02:34.646642 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: spawned uWSGI master process (pid: 81761) Sep 07 16:02:34.646846 dv-cinder-856174-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 07 16:02:34.651813 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: spawned uWSGI worker 1 (pid: 81775, cores: 1) Sep 07 16:02:34.651813 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: spawned uWSGI worker 2 (pid: 81776, cores: 1) Sep 07 16:02:34.651813 dv-cinder-856174-1-iscsi devstack@keystone.service[81761]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 07 16:02:38.986249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 16:02:38.986249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: from cryptography.utils import int_from_bytes Sep 07 16:02:38.987035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 16:02:38.987035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: from cryptography.utils import int_from_bytes Sep 07 16:02:38.988265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 16:02:38.988265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: from cryptography.utils import int_from_bytes Sep 07 16:02:38.988727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 16:02:38.988727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: from cryptography.utils import int_from_bytes Sep 07 16:02:39.039936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 16:02:39.040107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 16:02:39.040197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 16:02:39.040286 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 16:02:39.040365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] command line args: [] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 16:02:39.040442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] command line args: [] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 16:02:39.040562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 16:02:39.040676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 16:02:39.040761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ================================================================================ {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 16:02:39.040864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ================================================================================ {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 16:02:39.040978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] admin_token = **** {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.041119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] admin_token = **** {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.041227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] config_dir = [] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.041366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] config_dir = [] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.041442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.041553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.041636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] config_source = [] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.041734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] config_source = [] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.041821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] debug = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.041898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] debug = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.041970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.042173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.042335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.042421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.042487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.042551 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.042703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.042795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.042950 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.043029 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.043180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] list_limit = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.043265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] list_limit = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.043368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] log_config_append = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.043469 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] log_config_append = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.043575 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.043681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.043807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] log_dir = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.043978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] log_dir = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.044137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] log_file = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.044311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] log_file = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.044492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.044660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.044834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.045009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.045185 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.045353 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.045532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.045715 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.045891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.046060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.046237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.046415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.046591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.046759 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.046759 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.047089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.047272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.047427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.047602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.047800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.047977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.048119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.048301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.048475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.048621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.048753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.048895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.049010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.049133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.049282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.049408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.049523 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.049650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] publish_errors = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.049763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] publish_errors = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.049875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.050020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.050201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.050323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.050443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.050561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.050672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.050844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.051054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.051170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.051317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] standard_threads = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.051441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] standard_threads = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.051560 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.051682 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.051830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.051953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.052067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.052179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.052290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] use_journal = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.052410 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] use_journal = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.052525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] use_json = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.052644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] use_json = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.052757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] use_stderr = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.052874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] use_stderr = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.052987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] use_syslog = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.053614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] use_syslog = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.053733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.053852 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 16:02:39.053994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.053994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.054294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.054294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.054522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.054522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.054741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.054852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.054973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.055098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.055211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.055329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.055448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.055571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.055693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.055821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.055936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.056047 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.056158 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.056280 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.056405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.056519 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.056631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.056742 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.056854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.057219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.057358 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.057484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.057603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.057717 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.057829 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.057941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.058060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.058174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.058287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.058398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.058509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.058621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.058744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.058857 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.058969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.059088 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.059261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.059261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.059486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.059486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.059704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.059851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.059967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.060085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.060274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.060395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.060498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.060591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.060683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.060782 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.060889 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.060991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.061085 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.061177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.061277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.061371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.061472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.061566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.061665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.061758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.061856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.061958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.062064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.062158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.062256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.062351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.062442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.062551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.062646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.062738 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.062829 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.062929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.063021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.063125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.063224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.063318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.063416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.063509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.063638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.063745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.063884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.063981 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.064075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.064179 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.064314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.064446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.064568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.064688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.064790 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.064902 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.065021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.065142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.065264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.065384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.065510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.065625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.065735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.065847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.065847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.066080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.066193 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.066305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.066416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.066528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.066648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.066762 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.066874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.066987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.067105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.067219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.067339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.067450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.067563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.067675 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.067836 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.067956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.068078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.068201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.068322 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.068436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.068436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.068688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.068688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.069426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.069541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.069654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.069767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.069879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.069879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.070107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.070221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.070335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.070335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.070546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.070658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.070777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.070891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.071003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.071114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.071226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.071339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.071527 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.071644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.071747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.071891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.072005 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.072117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.072244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.072361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.072481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.072595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.072708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.072829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.072953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.073083 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.073083 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.073297 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.073411 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.073524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.073643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.073768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.073889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.074004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.074117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.074236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.074350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.074463 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.074582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.074703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.074823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.074944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.075058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.075180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.075295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.075419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.075535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.075648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.075808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.075939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.076056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.076176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.076291 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.076404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.076517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.076629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.076788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.076904 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.077017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.077130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.077242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.077364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.077479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.077591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.077704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.077817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.077929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.078054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.078168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.078280 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.078393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.078505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.078617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.078739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.078853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.078973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.079087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] auth.external = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.079199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.079342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.079466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.079581 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.079705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.079844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.079960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] auth.external = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.080085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] auth.password = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.080201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.080316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] auth.token = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.080438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.080544 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.080640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.080734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.080836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] auth.password = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.080940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.081037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] auth.token = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.081137 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.081234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.081329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.081424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.081526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.081623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.081718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.081821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.081917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] credential.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.082012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.082115 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.082212 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.082307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.082402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.082496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.082590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] credential.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.082693 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.082790 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.082891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.082891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.083076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.083171 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.083274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.083370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.083464 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.083560 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.083655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.083749 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.087741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.087741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.087983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] federation.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088925 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.088995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] federation.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.089946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090143 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090222 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090433 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.090937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091015 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091088 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.091976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092116 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.092987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093207 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.093975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.094054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.094128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.094199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.094259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.094438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.094533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.094616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.094690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.094762 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.094881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.094957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095464 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.095971 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096257 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096551 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.096969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097040 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097693 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.097985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098282 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.098992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.099172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.099255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.099400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.099478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.099636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.099724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.099918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.099998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.100159 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.100247 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.100390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.100477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.100636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.100715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.100880 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.100958 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.101112 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.101198 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.101347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.101427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.101600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.101740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.101822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.101907 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.102049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.102136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.102283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.102362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.102518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.102605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.102754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.102834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.102988 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.103066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.103216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.103302 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.103461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.103540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.103693 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.103792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.103953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.104054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.104185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.104263 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.104417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.104494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.104651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.104800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.104877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.104949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.105109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.105187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.105352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.105438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.105581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.105658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.105811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.105888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.106044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.106129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.106278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.106355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] resource.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.106513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.106592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.106756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.106841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.106993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.107072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.107222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.107299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.107457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.107543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.107697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] resource.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.107795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.107960 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.108038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.108194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.108283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.108429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.108506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.108661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] role.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.108738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.108906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] role.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.108992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.109135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.109218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.109378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.109454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.109613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.109699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.109849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.109927 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.110087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] role.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.110206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.110328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.110415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] role.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.110555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.110632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.110800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.110877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.111045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.111130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.111272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.111349 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.111515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.111598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.111756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.111893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.112018 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.112096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.112259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.112336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.112505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.112646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.112723 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.112878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.112955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.113128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.113214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.113386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.113591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.113704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.113785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.113868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.114073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.114206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.114337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.114466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.114594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.114722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.114859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.114988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.115117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.115254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.115384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.116081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.116178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.116254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.116325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.116397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.116469 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.116540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.116698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.116779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.116934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.117012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.117169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.117247 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.117397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.117478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.117632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] token.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.117714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.117869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.117956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.118121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.118200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.118356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.118434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.118591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.118677 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.118821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.118946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.119064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.119160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] token.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.119284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.119399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.119523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.119629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.119791 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.119943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.120049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.133441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.133577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.133655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.133727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.133798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.133868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.133953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.connection = **** {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.134987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135066 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135373 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.connection = **** {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.135854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136764 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136913 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.136913 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137250 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81775) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 16:02:39.137473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 16:02:39.137473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81776) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 16:02:39.155376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.155606 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.155861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.156181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.156415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.156644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.164147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.164935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.196571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.196844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.197233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.197467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.263206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.263655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.311607 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.311856 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.316233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.316506 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.467349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.467576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.467702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.467810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.467965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.468053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.468229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.468357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.468442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.468520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.468662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.468952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.497946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.498180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.498279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.498408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.498494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.498632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.498715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.498854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.498950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.499073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.499157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.499292 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.499373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.499542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.510289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.510819 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.531268 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.531444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.531553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.531667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.531812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.531899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.532038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.532120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.532263 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.532347 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.532484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.532564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81775) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.532702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.532913 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81776) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 16:02:39.549426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.549633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 16:02:39.549973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.550170 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 16:02:39.550440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.550535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.550664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 16:02:39.550746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.550946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.551027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.551189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 16:02:39.551279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.551464 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.551754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.552085 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.552396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.561627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.561905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.562177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.562493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.563176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.565437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.565437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.565437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.569422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.570928 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.572314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.572602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.572892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.573175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.573457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.573717 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.573807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.573986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.574068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.574257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.574345 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.574529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.574611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.574802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.574884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.575070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.575150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.575385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.575661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.575959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.576234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.576564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.597381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.598833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.599115 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.599589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.601613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.601896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.604285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.604567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.604850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.606722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.607010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.607295 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.611081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.611368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.613781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.614065 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.617666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.617960 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.618248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.618538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.618819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.619099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.619375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.619646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.620491 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.620790 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.621075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.621357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.621630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.621902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.622214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.622511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.635045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.635324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.638340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.638620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.639087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.642271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.642512 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.642597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.645798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.646086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.646661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.649749 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.650272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.652904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.653201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.653457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.653553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.653760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.654040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.654321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.656671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.656968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.657292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.657684 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.657966 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.658244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.670245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.670528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.670794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.671057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.674639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.674936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.675204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.675470 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.677849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.679946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.681469 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.681739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.683619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.684629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.684925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.685284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.685445 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.685654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.686269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.686847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.687125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.689855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.690139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.690421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.690853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.691259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.702014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.702303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.702586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.702721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.702866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.703134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.703250 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.703428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.703542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.703715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.703852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.704124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.704408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.704696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.719628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.720509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.722534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.723510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.726065 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.726977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.727567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.728604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.729102 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.729385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.729781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.730172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.730452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.730854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.736091 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.737336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.737774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.738791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.740689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.741083 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.741368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.741648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.741740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.742141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.742427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.742704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.749351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.749653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.749931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.750695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.751071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.751352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.756763 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81775) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.757048 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.757326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.757611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.757886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.758161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.758254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81776) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 16:02:39.758434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.758515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.758710 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.758800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.758979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.759061 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.759275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.759355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.759546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.759625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.759831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.759916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.760106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.760186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.760376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81775) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.760509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.760807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.761081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.761355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.761620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.761887 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81776) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 16:02:39.794969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a49b08f9c0 pid: 81775 (default app) Sep 07 16:02:39.797174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a49b08f9c0 pid: 81776 (default app) Sep 07 16:02:39.798223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201e63d-a717-402a-9ca5-0b267fb4992e None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:39.798507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201e63d-a717-402a-9ca5-0b267fb4992e None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:39.798775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201e63d-a717-402a-9ca5-0b267fb4992e None None] PATH_INFO: `/v3` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:39.802896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Wed Sep 7 16:02:39 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:02:47.384638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84103303-10b9-4dd7-b4d0-68d84dc14b4c None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:47.385129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84103303-10b9-4dd7-b4d0-68d84dc14b4c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:47.385209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84103303-10b9-4dd7-b4d0-68d84dc14b4c None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:47.389487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:02:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:02:47.394726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97d82f-c42a-407c-b0ed-a7342eb2513b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:47.395174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97d82f-c42a-407c-b0ed-a7342eb2513b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:47.395267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97d82f-c42a-407c-b0ed-a7342eb2513b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:47.465370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: INFO dbcounter [None req-fa97d82f-c42a-407c-b0ed-a7342eb2513b None None] Registered counter for database keystone Sep 07 16:02:47.486362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG oslo_db.sqlalchemy.engines [None req-fa97d82f-c42a-407c-b0ed-a7342eb2513b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81775) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 16:02:47.492508 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writer thread running {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 16:02:47.670864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG passlib.handlers.bcrypt [None req-fa97d82f-c42a-407c-b0ed-a7342eb2513b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81775) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 16:02:47.671188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG passlib.handlers.bcrypt [None req-fa97d82f-c42a-407c-b0ed-a7342eb2513b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81775) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 16:02:47.729369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-fa97d82f-c42a-407c-b0ed-a7342eb2513b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:47.731094 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fa97d82f-c42a-407c-b0ed-a7342eb2513b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:47.859019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:02:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 16:02:48.317759 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84103303-10b9-4dd7-b4d0-68d84dc14b4c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:48.387915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: INFO dbcounter [None req-84103303-10b9-4dd7-b4d0-68d84dc14b4c None None] Registered counter for database keystone Sep 07 16:02:48.406504 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG oslo_db.sqlalchemy.engines [None req-84103303-10b9-4dd7-b4d0-68d84dc14b4c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81776) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 16:02:48.413213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writer thread running {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 16:02:48.530475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84103303-10b9-4dd7-b4d0-68d84dc14b4c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:48.534682 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352826f8-964e-4458-b695-5a25568e4c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:48.535293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352826f8-964e-4458-b695-5a25568e4c39 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:48.535570 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352826f8-964e-4458-b695-5a25568e4c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:48.535865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352826f8-964e-4458-b695-5a25568e4c39 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:48.544567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352826f8-964e-4458-b695-5a25568e4c39 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:48.595390 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: WARNING keystone.common.rbac_enforcer.enforcer [None req-352826f8-964e-4458-b695-5a25568e4c39 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 16:02:48.595681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352826f8-964e-4458-b695-5a25568e4c39 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:48.605649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Sep 7 16:02:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 16:02:48.611643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97d82f-c42a-407c-b0ed-a7342eb2513b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:48.626578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97d82f-c42a-407c-b0ed-a7342eb2513b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:48.630994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bcf79-db30-42a6-a05e-a52246e6b7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:48.632120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bcf79-db30-42a6-a05e-a52246e6b7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:48.632401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bcf79-db30-42a6-a05e-a52246e6b7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:48.632686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bcf79-db30-42a6-a05e-a52246e6b7c2 admin admin] PATH_INFO: `/v3/projects/4f97db8bc2c2455d85720c36a1ed180d` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:48.634256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8bcf79-db30-42a6-a05e-a52246e6b7c2 admin admin] RBAC: Authorizing `identity:get_project(project_id=4f97db8bc2c2455d85720c36a1ed180d)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:48.687356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2d8bcf79-db30-42a6-a05e-a52246e6b7c2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 16:02:48.687700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8bcf79-db30-42a6-a05e-a52246e6b7c2 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:48.692287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Wed Sep 7 16:02:48 2022] GET /identity/v3/projects/4f97db8bc2c2455d85720c36a1ed180d => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:02:50.805418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635b5ee-034d-48d5-9e4d-bae8602819af None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:50.805930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635b5ee-034d-48d5-9e4d-bae8602819af None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:50.806011 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635b5ee-034d-48d5-9e4d-bae8602819af None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:50.810329 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:02:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:02:50.815783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c37206-fd81-48fc-94de-9bdb2a92fe9e None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:50.816140 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c37206-fd81-48fc-94de-9bdb2a92fe9e None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:50.816355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c37206-fd81-48fc-94de-9bdb2a92fe9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:50.837103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-57c37206-fd81-48fc-94de-9bdb2a92fe9e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:50.838614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-57c37206-fd81-48fc-94de-9bdb2a92fe9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:50.846402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:02:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 16:02:51.113290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635b5ee-034d-48d5-9e4d-bae8602819af None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:51.125819 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635b5ee-034d-48d5-9e4d-bae8602819af None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:51.129967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f58ae-82f6-43b4-bb97-ea8cbc61697b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:51.131160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f58ae-82f6-43b4-bb97-ea8cbc61697b admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:51.131447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f58ae-82f6-43b4-bb97-ea8cbc61697b admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:51.131724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f58ae-82f6-43b4-bb97-ea8cbc61697b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:51.139318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f58ae-82f6-43b4-bb97-ea8cbc61697b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:51.141119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f58ae-82f6-43b4-bb97-ea8cbc61697b admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:51.149785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Wed Sep 7 16:02:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:02:51.154986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c37206-fd81-48fc-94de-9bdb2a92fe9e None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:51.164153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c37206-fd81-48fc-94de-9bdb2a92fe9e None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:51.168543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461a406-ac14-40eb-9d96-cbf112e71186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:51.169163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461a406-ac14-40eb-9d96-cbf112e71186 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:51.169444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461a406-ac14-40eb-9d96-cbf112e71186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:51.169719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461a406-ac14-40eb-9d96-cbf112e71186 admin admin] PATH_INFO: `/v3/users/f2181a60abf646d28f09241ba111a2d3` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:51.171028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461a406-ac14-40eb-9d96-cbf112e71186 admin admin] RBAC: Authorizing `identity:get_user(user_id=f2181a60abf646d28f09241ba111a2d3)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:51.172890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461a406-ac14-40eb-9d96-cbf112e71186 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:51.174401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Wed Sep 7 16:02:51 2022] GET /identity/v3/users/f2181a60abf646d28f09241ba111a2d3 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:02:53.390849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f11dbd-d166-4d93-82d6-0be07aeb3b83 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:53.391405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f11dbd-d166-4d93-82d6-0be07aeb3b83 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:53.391405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f11dbd-d166-4d93-82d6-0be07aeb3b83 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:53.392914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:02:53.397771 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edfaa3-aa97-4908-bf10-1ee98821763a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:53.398165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edfaa3-aa97-4908-bf10-1ee98821763a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:53.398324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edfaa3-aa97-4908-bf10-1ee98821763a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:53.414662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ebc2-2424-445e-a6e7-c93fd2f9d898 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:53.414942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ebc2-2424-445e-a6e7-c93fd2f9d898 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:53.415329 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35ebc2-2424-445e-a6e7-c93fd2f9d898 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:53.415954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-57edfaa3-aa97-4908-bf10-1ee98821763a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:53.416650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:02:53.417123 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-57edfaa3-aa97-4908-bf10-1ee98821763a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:53.421671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82db7d1-1976-4272-9cd5-826defce1969 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:53.421950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82db7d1-1976-4272-9cd5-826defce1969 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:53.422222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82db7d1-1976-4272-9cd5-826defce1969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:53.464208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG passlib.handlers.bcrypt [None req-a82db7d1-1976-4272-9cd5-826defce1969 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81776) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 16:02:53.464593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG passlib.handlers.bcrypt [None req-a82db7d1-1976-4272-9cd5-826defce1969 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81776) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 16:02:53.496165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:02:53.513901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-a82db7d1-1976-4272-9cd5-826defce1969 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:53.515498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a82db7d1-1976-4272-9cd5-826defce1969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:53.613346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:02:53.774382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edfaa3-aa97-4908-bf10-1ee98821763a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:53.799143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:53.877241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edfaa3-aa97-4908-bf10-1ee98821763a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:53.893428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82db7d1-1976-4272-9cd5-826defce1969 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:53.914928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae46f5-74a2-43a6-ae0a-d130ddd47ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:53.915569 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae46f5-74a2-43a6-ae0a-d130ddd47ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:53.915877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae46f5-74a2-43a6-ae0a-d130ddd47ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:53.916155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae46f5-74a2-43a6-ae0a-d130ddd47ce7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:53.926844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae46f5-74a2-43a6-ae0a-d130ddd47ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-7dae46f5-74a2-43a6-ae0a-d130ddd47ce7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:02:53.936403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:02:53.937325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:02:53.938256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:02:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:02:53.943187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae46f5-74a2-43a6-ae0a-d130ddd47ce7 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:53.997563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82db7d1-1976-4272-9cd5-826defce1969 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:54.000010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:54.032767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:54.036135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3357a82-fd2a-43f1-b1cf-50aece5f4d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:54.037610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357a82-fd2a-43f1-b1cf-50aece5f4d01 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:54.037898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357a82-fd2a-43f1-b1cf-50aece5f4d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:54.038172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357a82-fd2a-43f1-b1cf-50aece5f4d01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:54.050786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae46f5-74a2-43a6-ae0a-d130ddd47ce7 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:54.084744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e14228-7f59-4654-aa97-93d4450fd0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:54.085351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e14228-7f59-4654-aa97-93d4450fd0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:54.085630 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e14228-7f59-4654-aa97-93d4450fd0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:54.085931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e14228-7f59-4654-aa97-93d4450fd0e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:54.086596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e14228-7f59-4654-aa97-93d4450fd0e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:02:54.086900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e14228-7f59-4654-aa97-93d4450fd0e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:54.088322 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e14228-7f59-4654-aa97-93d4450fd0e8 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:54.100435 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:02:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:02:54.105259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3357a82-fd2a-43f1-b1cf-50aece5f4d01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:54.106106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e14228-7f59-4654-aa97-93d4450fd0e8 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:54.106801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3357a82-fd2a-43f1-b1cf-50aece5f4d01 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:54.200975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e14228-7f59-4654-aa97-93d4450fd0e8 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:54.202196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:02:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 16:02:54.207581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3357a82-fd2a-43f1-b1cf-50aece5f4d01 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:54.210727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:54.235734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71505906-74a0-49a9-a90f-f07cb29bcec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:54.236390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71505906-74a0-49a9-a90f-f07cb29bcec7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:54.236666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71505906-74a0-49a9-a90f-f07cb29bcec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:54.236939 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71505906-74a0-49a9-a90f-f07cb29bcec7 None admin] PATH_INFO: `/v3/users/f2181a60abf646d28f09241ba111a2d3` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:54.238055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71505906-74a0-49a9-a90f-f07cb29bcec7 None admin] RBAC: Authorizing `identity:get_user(user_id=f2181a60abf646d28f09241ba111a2d3)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:54.239563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71505906-74a0-49a9-a90f-f07cb29bcec7 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:54.241143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:02:54 2022] GET /identity/v3/users/f2181a60abf646d28f09241ba111a2d3 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:02:54.246375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71505906-74a0-49a9-a90f-f07cb29bcec7 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:54.261614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:54.307599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3357a82-fd2a-43f1-b1cf-50aece5f4d01 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:54.340745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948b4f6-4b91-460c-bb84-6c1e18bad34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:54.341595 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948b4f6-4b91-460c-bb84-6c1e18bad34f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:54.341875 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948b4f6-4b91-460c-bb84-6c1e18bad34f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:54.342152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948b4f6-4b91-460c-bb84-6c1e18bad34f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:54.344852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71505906-74a0-49a9-a90f-f07cb29bcec7 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:54.361114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948b4f6-4b91-460c-bb84-6c1e18bad34f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:54.362579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948b4f6-4b91-460c-bb84-6c1e18bad34f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:54.377321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3baba4-83ed-4db4-9caa-acf366cf1459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:54.377941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3baba4-83ed-4db4-9caa-acf366cf1459 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:54.378218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3baba4-83ed-4db4-9caa-acf366cf1459 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:54.378489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3baba4-83ed-4db4-9caa-acf366cf1459 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:54.379600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3baba4-83ed-4db4-9caa-acf366cf1459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-9948b4f6-4b91-460c-bb84-6c1e18bad34f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:02:54.380089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:02:54.381600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:02:54.382564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:02:54.382564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:02:54.382564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:02:54.382725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3baba4-83ed-4db4-9caa-acf366cf1459 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:54.383084 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 9/18] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:02:54.383176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 10/19] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:02:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:02:54.387496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948b4f6-4b91-460c-bb84-6c1e18bad34f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:54.392002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3baba4-83ed-4db4-9caa-acf366cf1459 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:54.442535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:54.470940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:54.486450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948b4f6-4b91-460c-bb84-6c1e18bad34f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:54.488961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3baba4-83ed-4db4-9caa-acf366cf1459 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:54.520184 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e1dda-1f23-456b-9e4a-c8c8eb44d46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:54.520785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e1dda-1f23-456b-9e4a-c8c8eb44d46f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:54.521665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e1dda-1f23-456b-9e4a-c8c8eb44d46f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:54.521940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e1dda-1f23-456b-9e4a-c8c8eb44d46f None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:54.522580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e1dda-1f23-456b-9e4a-c8c8eb44d46f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:02:54.522885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e1dda-1f23-456b-9e4a-c8c8eb44d46f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:54.523330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac2362-fa43-4f08-8f43-67b3ee8a40d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:54.523970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac2362-fa43-4f08-8f43-67b3ee8a40d1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:54.524249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac2362-fa43-4f08-8f43-67b3ee8a40d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:54.524347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e1dda-1f23-456b-9e4a-c8c8eb44d46f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:54.524521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac2362-fa43-4f08-8f43-67b3ee8a40d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:54.525238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac2362-fa43-4f08-8f43-67b3ee8a40d1 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.domain.id=default, user.id=f2181a60abf646d28f09241ba111a2d3) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:02:54.525544 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac2362-fa43-4f08-8f43-67b3ee8a40d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:54.526963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac2362-fa43-4f08-8f43-67b3ee8a40d1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:54.533578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Wed Sep 7 16:02:54 2022] GET /identity/v3/role_assignments?user.id=f2181a60abf646d28f09241ba111a2d3&scope.domain.id=default&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:02:54.541782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:02:56.967142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875adf-e048-4c77-8330-e84964f0be5a None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:56.968189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875adf-e048-4c77-8330-e84964f0be5a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:56.968766 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875adf-e048-4c77-8330-e84964f0be5a None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:56.970298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:02:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:02:56.975864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5da6e-e323-42d7-95f2-a0c14ac90110 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:56.976674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5da6e-e323-42d7-95f2-a0c14ac90110 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:56.977144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5da6e-e323-42d7-95f2-a0c14ac90110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:57.016820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-5de5da6e-e323-42d7-95f2-a0c14ac90110 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:57.018304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-5de5da6e-e323-42d7-95f2-a0c14ac90110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:57.106116 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 11/23] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:02:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:02:57.504084 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e43a0-fb7a-49c4-afbe-84c1180d0cbf None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:57.504745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e43a0-fb7a-49c4-afbe-84c1180d0cbf None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:57.505274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e43a0-fb7a-49c4-afbe-84c1180d0cbf None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:57.506819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 13/24] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:02:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:02:57.517245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ccbb-fd81-42e5-a29c-9d789a4b0ed0 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:57.517775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ccbb-fd81-42e5-a29c-9d789a4b0ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:57.518250 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ccbb-fd81-42e5-a29c-9d789a4b0ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:57.548726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-1020ccbb-fd81-42e5-a29c-9d789a4b0ed0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:57.550223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1020ccbb-fd81-42e5-a29c-9d789a4b0ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:57.577726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b82d2-db5a-475d-ad10-c2911c7d7dda None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:57.578596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b82d2-db5a-475d-ad10-c2911c7d7dda None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:57.579362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b82d2-db5a-475d-ad10-c2911c7d7dda None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:57.581423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 14/25] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:02:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:02:57.582597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b82d2-db5a-475d-ad10-c2911c7d7dda None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:57.605388 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:57.633927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 12/26] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:02:57.635308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cd112-5920-46d6-94dc-70595c56d696 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:57.635605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cd112-5920-46d6-94dc-70595c56d696 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:57.635912 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527cd112-5920-46d6-94dc-70595c56d696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:57.653929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-527cd112-5920-46d6-94dc-70595c56d696 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:57.655298 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-527cd112-5920-46d6-94dc-70595c56d696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:57.748574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b82d2-db5a-475d-ad10-c2911c7d7dda None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:57.775068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 13/27] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:02:57.776697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067623b-0b92-48e4-989d-36a9263b0725 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:57.777164 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067623b-0b92-48e4-989d-36a9263b0725 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:57.777611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067623b-0b92-48e4-989d-36a9263b0725 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:57.779037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 14/28] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:02:57.784573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26be22-23e5-4f11-b2f0-ef5e27eef79f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:57.785078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26be22-23e5-4f11-b2f0-ef5e27eef79f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:57.785526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26be22-23e5-4f11-b2f0-ef5e27eef79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:57.815507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca492e0-769a-4afc-a98f-dc60efe2ac1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:57.816492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca492e0-769a-4afc-a98f-dc60efe2ac1f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:57.816998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca492e0-769a-4afc-a98f-dc60efe2ac1f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:57.817473 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca492e0-769a-4afc-a98f-dc60efe2ac1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:57.818961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-8d26be22-23e5-4f11-b2f0-ef5e27eef79f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:57.821658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8d26be22-23e5-4f11-b2f0-ef5e27eef79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:57.839474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca492e0-769a-4afc-a98f-dc60efe2ac1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-7ca492e0-769a-4afc-a98f-dc60efe2ac1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:02:57.845370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:02:57.846196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:02:57.847064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:02:57.853438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 15/29] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:02:57.860003 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca492e0-769a-4afc-a98f-dc60efe2ac1f None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:57.866993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:57.951999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 15/30] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:02:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:02:57.952938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26be22-23e5-4f11-b2f0-ef5e27eef79f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:58.025255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca492e0-769a-4afc-a98f-dc60efe2ac1f None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:58.048863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:58.082916 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47994e-2fa9-48f9-8724-329216dabc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:58.084160 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47994e-2fa9-48f9-8724-329216dabc69 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:58.084442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47994e-2fa9-48f9-8724-329216dabc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:58.084714 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47994e-2fa9-48f9-8724-329216dabc69 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:58.085387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47994e-2fa9-48f9-8724-329216dabc69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:02:58.085695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47994e-2fa9-48f9-8724-329216dabc69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:58.087121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47994e-2fa9-48f9-8724-329216dabc69 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:58.096201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 16/31] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:02:58.102509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47994e-2fa9-48f9-8724-329216dabc69 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:58.181584 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:58.188520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26be22-23e5-4f11-b2f0-ef5e27eef79f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:58.225972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47994e-2fa9-48f9-8724-329216dabc69 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:58.261362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77aa00-a448-4d3f-a3aa-92320c17153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:58.262208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77aa00-a448-4d3f-a3aa-92320c17153b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:58.262734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77aa00-a448-4d3f-a3aa-92320c17153b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:58.263170 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77aa00-a448-4d3f-a3aa-92320c17153b None admin] PATH_INFO: `/v3/users/f2181a60abf646d28f09241ba111a2d3` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:58.264573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77aa00-a448-4d3f-a3aa-92320c17153b None admin] RBAC: Authorizing `identity:get_user(user_id=f2181a60abf646d28f09241ba111a2d3)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:58.266184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77aa00-a448-4d3f-a3aa-92320c17153b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:58.268236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 17/32] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:02:58 2022] GET /identity/v3/users/f2181a60abf646d28f09241ba111a2d3 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:02:58.269136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77aa00-a448-4d3f-a3aa-92320c17153b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:58.288547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f8ab0-bc0b-4e79-bbc4-9a6d17569fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:58.289176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f8ab0-bc0b-4e79-bbc4-9a6d17569fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:58.289460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f8ab0-bc0b-4e79-bbc4-9a6d17569fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:58.289780 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f8ab0-bc0b-4e79-bbc4-9a6d17569fe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:58.320562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f8ab0-bc0b-4e79-bbc4-9a6d17569fe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:58.322281 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f8ab0-bc0b-4e79-bbc4-9a6d17569fe7 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-4e1f8ab0-bc0b-4e79-bbc4-9a6d17569fe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:02:58.356978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:02:58.358108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:02:58.359132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:02:58.359132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:02:58.359132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:02:58.360854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 16/33] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:02:58.361804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f8ab0-bc0b-4e79-bbc4-9a6d17569fe7 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:58.387494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77aa00-a448-4d3f-a3aa-92320c17153b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:58.437467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa003-bcb5-45d4-b863-47a6ebd674c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:58.438319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa003-bcb5-45d4-b863-47a6ebd674c1 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:58.438844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa003-bcb5-45d4-b863-47a6ebd674c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:58.439306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149aa003-bcb5-45d4-b863-47a6ebd674c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:58.440324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aa003-bcb5-45d4-b863-47a6ebd674c1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:58.441838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149aa003-bcb5-45d4-b863-47a6ebd674c1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:58.445964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:58.471326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:58.484944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f8ab0-bc0b-4e79-bbc4-9a6d17569fe7 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:58.518547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c1b97-be9d-4b2b-81c7-c61b5642ab30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:58.519182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1b97-be9d-4b2b-81c7-c61b5642ab30 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:58.519467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1b97-be9d-4b2b-81c7-c61b5642ab30 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:58.519734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1b97-be9d-4b2b-81c7-c61b5642ab30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:58.521687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c1b97-be9d-4b2b-81c7-c61b5642ab30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:58.523127 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c1b97-be9d-4b2b-81c7-c61b5642ab30 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:58.524706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 17/34] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:02:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:02:58.525444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c1b97-be9d-4b2b-81c7-c61b5642ab30 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:58.628396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c1b97-be9d-4b2b-81c7-c61b5642ab30 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:58.655990 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 18/35] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Sep 7 16:02:58 2022] POST /identity/v3/roles => generated 225 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:02:58.655990 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa003-bcb5-45d4-b863-47a6ebd674c1 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:58.675511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207df723-aa65-49e1-ad63-997132d45164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:58.676280 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207df723-aa65-49e1-ad63-997132d45164 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:58.676593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207df723-aa65-49e1-ad63-997132d45164 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:58.677057 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207df723-aa65-49e1-ad63-997132d45164 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:58.677757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207df723-aa65-49e1-ad63-997132d45164 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:58.680170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207df723-aa65-49e1-ad63-997132d45164 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:58.780855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149aa003-bcb5-45d4-b863-47a6ebd674c1 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:58.830040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b4c388-834d-451b-a139-a5d8a573d591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:58.830697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4c388-834d-451b-a139-a5d8a573d591 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:58.831621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4c388-834d-451b-a139-a5d8a573d591 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:58.831621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b4c388-834d-451b-a139-a5d8a573d591 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:58.832035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4c388-834d-451b-a139-a5d8a573d591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:02:58.832346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4c388-834d-451b-a139-a5d8a573d591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:58.834443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b4c388-834d-451b-a139-a5d8a573d591 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:58.855864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 19/36] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:02:58.855864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c44c6b-ee52-46e5-ac9e-c9eb79676f47 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:58.858237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c44c6b-ee52-46e5-ac9e-c9eb79676f47 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:58.858237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c44c6b-ee52-46e5-ac9e-c9eb79676f47 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:58.858424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 20/37] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:02:58.859387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c44c6b-ee52-46e5-ac9e-c9eb79676f47 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:58.886861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 18/38] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Sep 7 16:02:58 2022] POST /identity/v3/roles => generated 219 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:02:58.887605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207df723-aa65-49e1-ad63-997132d45164 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:58.910108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:58.944135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:58.997248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c44c6b-ee52-46e5-ac9e-c9eb79676f47 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:59.023131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207df723-aa65-49e1-ad63-997132d45164 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:59.038483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42c2c4-70a9-4a0a-94e8-83052f63fbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:59.039289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42c2c4-70a9-4a0a-94e8-83052f63fbdc None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:59.039576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42c2c4-70a9-4a0a-94e8-83052f63fbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:59.039878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42c2c4-70a9-4a0a-94e8-83052f63fbdc None admin] PATH_INFO: `/v3/domains/default/users/f2181a60abf646d28f09241ba111a2d3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:59.042136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42c2c4-70a9-4a0a-94e8-83052f63fbdc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f2181a60abf646d28f09241ba111a2d3, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:59.043615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42c2c4-70a9-4a0a-94e8-83052f63fbdc None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:59.062239 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd93a5f-75f8-4a95-8cc0-b8c43634b0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:59.062882 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93a5f-75f8-4a95-8cc0-b8c43634b0ea None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:59.063168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93a5f-75f8-4a95-8cc0-b8c43634b0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:59.063447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd93a5f-75f8-4a95-8cc0-b8c43634b0ea None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:59.064155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd93a5f-75f8-4a95-8cc0-b8c43634b0ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:59.065824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd93a5f-75f8-4a95-8cc0-b8c43634b0ea None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:59.140301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 21/39] 10.222.0.15 () {58 vars in 1361 bytes} [Wed Sep 7 16:02:58 2022] PUT /identity/v3/domains/default/users/f2181a60abf646d28f09241ba111a2d3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:02:59.141653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48b601-8410-44f7-9c7b-797724e051c1 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:59.141947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48b601-8410-44f7-9c7b-797724e051c1 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:59.142227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48b601-8410-44f7-9c7b-797724e051c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:59.163922 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-5c48b601-8410-44f7-9c7b-797724e051c1 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:59.165555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5c48b601-8410-44f7-9c7b-797724e051c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:59.171820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 19/40] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:02:58 2022] POST /identity/v3/projects => generated 301 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:02:59.173490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2575fe-8c7d-4f66-837f-cef73a1f6cc6 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:59.174215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2575fe-8c7d-4f66-837f-cef73a1f6cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:59.174690 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2575fe-8c7d-4f66-837f-cef73a1f6cc6 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:59.176381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 20/41] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:02:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:02:59.185050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c88f8-79bc-4b27-9f77-6fa5457b253f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:59.185714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c88f8-79bc-4b27-9f77-6fa5457b253f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:59.186186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c88f8-79bc-4b27-9f77-6fa5457b253f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:59.212008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-6f1c88f8-79bc-4b27-9f77-6fa5457b253f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:59.213820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6f1c88f8-79bc-4b27-9f77-6fa5457b253f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:59.297988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:59.312435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 21/42] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:02:59.381965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 22/43] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:02:59.583021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c88f8-79bc-4b27-9f77-6fa5457b253f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:59.647405 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48b601-8410-44f7-9c7b-797724e051c1 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:59.688147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c88f8-79bc-4b27-9f77-6fa5457b253f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:59.699232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:59.722219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b6b4b-5ff6-43eb-b0a9-285f7c327504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:59.722851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b6b4b-5ff6-43eb-b0a9-285f7c327504 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:59.723134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b6b4b-5ff6-43eb-b0a9-285f7c327504 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:59.723408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b6b4b-5ff6-43eb-b0a9-285f7c327504 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:59.725115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b6b4b-5ff6-43eb-b0a9-285f7c327504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:59.726565 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b6b4b-5ff6-43eb-b0a9-285f7c327504 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:59.728369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 22/44] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:02:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:02:59.733437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b6b4b-5ff6-43eb-b0a9-285f7c327504 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:59.757022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48b601-8410-44f7-9c7b-797724e051c1 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:59.799272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:59.801297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb5ffc-fd48-4bc6-8eb2-3aae15edb05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:59.801932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb5ffc-fd48-4bc6-8eb2-3aae15edb05a None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:59.804647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb5ffc-fd48-4bc6-8eb2-3aae15edb05a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:59.804647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb5ffc-fd48-4bc6-8eb2-3aae15edb05a None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:59.804647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb5ffc-fd48-4bc6-8eb2-3aae15edb05a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:59.804647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb5ffc-fd48-4bc6-8eb2-3aae15edb05a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:59.835007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b6b4b-5ff6-43eb-b0a9-285f7c327504 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:59.869583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74696d-0abf-4e77-a229-b16b90580329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:59.870190 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74696d-0abf-4e77-a229-b16b90580329 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:59.870469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74696d-0abf-4e77-a229-b16b90580329 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:59.870740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74696d-0abf-4e77-a229-b16b90580329 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:59.871383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74696d-0abf-4e77-a229-b16b90580329 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:59.872865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74696d-0abf-4e77-a229-b16b90580329 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:59.909138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 23/45] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Sep 7 16:02:59 2022] POST /identity/v3/roles => generated 223 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:02:59.963930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 23/46] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:02:59 2022] POST /identity/v3/projects => generated 312 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:03:01.356118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff87ff4-2ac2-41f1-8809-95da468426a2 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:01.356597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff87ff4-2ac2-41f1-8809-95da468426a2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:01.356597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff87ff4-2ac2-41f1-8809-95da468426a2 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:01.357925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 24/47] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:01.363384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e089d1-fbcb-4dcd-9cf4-d725240395f5 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:01.363837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e089d1-fbcb-4dcd-9cf4-d725240395f5 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:01.363975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e089d1-fbcb-4dcd-9cf4-d725240395f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:01.381565 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-25e089d1-fbcb-4dcd-9cf4-d725240395f5 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:01.382769 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-25e089d1-fbcb-4dcd-9cf4-d725240395f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:01.446935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 24/48] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:01.714094 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff87ff4-2ac2-41f1-8809-95da468426a2 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:01.821610 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff87ff4-2ac2-41f1-8809-95da468426a2 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:01.823959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:01.854791 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb69358-650e-46e8-bead-639a06035aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:01.855423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb69358-650e-46e8-bead-639a06035aae None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:01.855736 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb69358-650e-46e8-bead-639a06035aae None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:01.856418 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb69358-650e-46e8-bead-639a06035aae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:01.867330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb69358-650e-46e8-bead-639a06035aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-deb69358-650e-46e8-bead-639a06035aae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:01.872230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:01.873043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:01.873811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 25/49] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:03:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:03:01.881069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e089d1-fbcb-4dcd-9cf4-d725240395f5 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:01.928474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:01.983648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e089d1-fbcb-4dcd-9cf4-d725240395f5 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:02.026770 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda0c1e-3d02-44b9-a340-909bfbe542cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:02.027402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda0c1e-3d02-44b9-a340-909bfbe542cd None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:02.027689 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda0c1e-3d02-44b9-a340-909bfbe542cd None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:02.028000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda0c1e-3d02-44b9-a340-909bfbe542cd None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:02.028731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda0c1e-3d02-44b9-a340-909bfbe542cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:02.029041 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda0c1e-3d02-44b9-a340-909bfbe542cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:02.030570 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda0c1e-3d02-44b9-a340-909bfbe542cd None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:02.120930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0553944-e48c-4ecd-a38c-555f704401e2 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:02.121201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0553944-e48c-4ecd-a38c-555f704401e2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:02.121475 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0553944-e48c-4ecd-a38c-555f704401e2 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:02.122856 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 26/50] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:03:02.128492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f26884-97d8-4596-9cdd-eede4201ff74 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:02.128777 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f26884-97d8-4596-9cdd-eede4201ff74 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:02.129055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f26884-97d8-4596-9cdd-eede4201ff74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:02.147126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 25/51] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:02.147688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-92f26884-97d8-4596-9cdd-eede4201ff74 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:02.148933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-92f26884-97d8-4596-9cdd-eede4201ff74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:02.152804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda0c1e-3d02-44b9-a340-909bfbe542cd None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:02.215996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 27/52] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:02.249960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda0c1e-3d02-44b9-a340-909bfbe542cd None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:02.260086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:02.281664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee425c-2ca6-4cd7-a171-085dbc9c6a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:02.282292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee425c-2ca6-4cd7-a171-085dbc9c6a41 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:02.282569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee425c-2ca6-4cd7-a171-085dbc9c6a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:02.282849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee425c-2ca6-4cd7-a171-085dbc9c6a41 None admin] PATH_INFO: `/v3/users/f2181a60abf646d28f09241ba111a2d3` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:02.284226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee425c-2ca6-4cd7-a171-085dbc9c6a41 None admin] RBAC: Authorizing `identity:get_user(user_id=f2181a60abf646d28f09241ba111a2d3)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:02.285665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee425c-2ca6-4cd7-a171-085dbc9c6a41 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:02.287755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 26/53] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:02 2022] GET /identity/v3/users/f2181a60abf646d28f09241ba111a2d3 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:02.293226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f26884-97d8-4596-9cdd-eede4201ff74 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:02.387599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:02.394312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f26884-97d8-4596-9cdd-eede4201ff74 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:02.428684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317188b-1baf-450d-b4da-ed278ac1f767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:02.429305 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317188b-1baf-450d-b4da-ed278ac1f767 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:02.429654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317188b-1baf-450d-b4da-ed278ac1f767 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:02.429928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317188b-1baf-450d-b4da-ed278ac1f767 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:02.431029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317188b-1baf-450d-b4da-ed278ac1f767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:02.432762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317188b-1baf-450d-b4da-ed278ac1f767 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:02.434309 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 28/54] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:02.440063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee425c-2ca6-4cd7-a171-085dbc9c6a41 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:02.482104 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317188b-1baf-450d-b4da-ed278ac1f767 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:02.538255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee425c-2ca6-4cd7-a171-085dbc9c6a41 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:02.571771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684994a-0a91-470d-ae30-62b50b6fdc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:02.572406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684994a-0a91-470d-ae30-62b50b6fdc9c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:02.572692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684994a-0a91-470d-ae30-62b50b6fdc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:02.572972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684994a-0a91-470d-ae30-62b50b6fdc9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:02.573700 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684994a-0a91-470d-ae30-62b50b6fdc9c None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.domain.id=default, user.id=f2181a60abf646d28f09241ba111a2d3) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:02.574010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684994a-0a91-470d-ae30-62b50b6fdc9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:02.575455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684994a-0a91-470d-ae30-62b50b6fdc9c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:02.581712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317188b-1baf-450d-b4da-ed278ac1f767 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:02.584107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 27/55] 10.222.0.15 () {56 vars in 1322 bytes} [Wed Sep 7 16:03:02 2022] GET /identity/v3/role_assignments?user.id=f2181a60abf646d28f09241ba111a2d3&scope.domain.id=default&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:02.618336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3ae6a-1f46-47bb-a021-6ef4100818a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:02.618993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ae6a-1f46-47bb-a021-6ef4100818a5 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:02.619272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ae6a-1f46-47bb-a021-6ef4100818a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:02.619543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3ae6a-1f46-47bb-a021-6ef4100818a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:02.620745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3ae6a-1f46-47bb-a021-6ef4100818a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:02.622255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3ae6a-1f46-47bb-a021-6ef4100818a5 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:02.623919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 29/56] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:02.629588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684994a-0a91-470d-ae30-62b50b6fdc9c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:02.676575 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:02.736144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684994a-0a91-470d-ae30-62b50b6fdc9c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:02.771284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb7370-1660-40d3-9ffa-dd5a0e1a365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:02.771945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb7370-1660-40d3-9ffa-dd5a0e1a365a None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:02.772231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb7370-1660-40d3-9ffa-dd5a0e1a365a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:02.772508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb7370-1660-40d3-9ffa-dd5a0e1a365a None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:02.773180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb7370-1660-40d3-9ffa-dd5a0e1a365a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:02.774628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb7370-1660-40d3-9ffa-dd5a0e1a365a None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:02.857698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 28/57] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Sep 7 16:03:02 2022] POST /identity/v3/projects => generated 298 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:03:04.979792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fd14e-1fe8-4877-9b85-4bcb331303a4 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:04.980340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fd14e-1fe8-4877-9b85-4bcb331303a4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:04.980340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fd14e-1fe8-4877-9b85-4bcb331303a4 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:04.981711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 30/58] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:04.987050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9294e-930a-4a2e-9ba0-14e0d982555f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:04.987463 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9294e-930a-4a2e-9ba0-14e0d982555f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:04.987603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9294e-930a-4a2e-9ba0-14e0d982555f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:05.007285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ccb9294e-930a-4a2e-9ba0-14e0d982555f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:05.008518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ccb9294e-930a-4a2e-9ba0-14e0d982555f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:05.075115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 29/59] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:05.345324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fd14e-1fe8-4877-9b85-4bcb331303a4 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:05.375961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:05.444920 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fd14e-1fe8-4877-9b85-4bcb331303a4 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:05.477388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc76246-72a3-48ad-a12d-4543180ac778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:05.478011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc76246-72a3-48ad-a12d-4543180ac778 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:05.478294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc76246-72a3-48ad-a12d-4543180ac778 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:05.478573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc76246-72a3-48ad-a12d-4543180ac778 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:05.479680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc76246-72a3-48ad-a12d-4543180ac778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:05.481149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc76246-72a3-48ad-a12d-4543180ac778 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:05.482675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 31/60] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:05.488055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9294e-930a-4a2e-9ba0-14e0d982555f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:05.511961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:05.583599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9294e-930a-4a2e-9ba0-14e0d982555f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:05.616317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31030752-9e9e-45ab-892d-e7d902d15657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:05.616905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030752-9e9e-45ab-892d-e7d902d15657 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:05.617184 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030752-9e9e-45ab-892d-e7d902d15657 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:05.617457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31030752-9e9e-45ab-892d-e7d902d15657 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:05.618099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31030752-9e9e-45ab-892d-e7d902d15657 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:05.619532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31030752-9e9e-45ab-892d-e7d902d15657 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:05.727967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 30/61] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:03:05 2022] POST /identity/v3/users => generated 275 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:03:08.710274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c13f-d7d0-41d2-990a-aaf525d2d6f6 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:08.710787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c13f-d7d0-41d2-990a-aaf525d2d6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:08.710859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803c13f-d7d0-41d2-990a-aaf525d2d6f6 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:08.712228 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 32/62] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:08.717751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c794d5-24be-4a74-aa10-247abfe9e1b3 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:08.718643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c794d5-24be-4a74-aa10-247abfe9e1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:08.719062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c794d5-24be-4a74-aa10-247abfe9e1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:08.748322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-75c794d5-24be-4a74-aa10-247abfe9e1b3 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:08.749793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-75c794d5-24be-4a74-aa10-247abfe9e1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:08.860120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 31/63] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:08.873933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e6e94-5bfc-4b73-b476-cdbdc712e51d None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:08.874512 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e6e94-5bfc-4b73-b476-cdbdc712e51d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:08.874989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e6e94-5bfc-4b73-b476-cdbdc712e51d None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:08.876659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 33/64] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:08.882646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682169a-edb6-4f1b-97b8-a3d0b09d7b32 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:08.883222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682169a-edb6-4f1b-97b8-a3d0b09d7b32 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:08.883715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682169a-edb6-4f1b-97b8-a3d0b09d7b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:08.903464 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-d682169a-edb6-4f1b-97b8-a3d0b09d7b32 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:08.904954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d682169a-edb6-4f1b-97b8-a3d0b09d7b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:08.936144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:08.980115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 32/65] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:09.165161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e6e94-5bfc-4b73-b476-cdbdc712e51d None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:09.291143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:09.300007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e6e94-5bfc-4b73-b476-cdbdc712e51d None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:09.349518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682169a-edb6-4f1b-97b8-a3d0b09d7b32 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:09.350782 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5597b8f-971a-4695-a244-c3b365825d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:09.351435 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5597b8f-971a-4695-a244-c3b365825d7c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:09.351719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5597b8f-971a-4695-a244-c3b365825d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:09.352020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5597b8f-971a-4695-a244-c3b365825d7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:09.381687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5597b8f-971a-4695-a244-c3b365825d7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-b5597b8f-971a-4695-a244-c3b365825d7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:09.390445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:09.391266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:09.392446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 34/66] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:09.398272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5597b8f-971a-4695-a244-c3b365825d7c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:09.489686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682169a-edb6-4f1b-97b8-a3d0b09d7b32 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:09.529096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a6e53-e846-46b1-8fea-17839e45900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:09.529804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a6e53-e846-46b1-8fea-17839e45900b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:09.530093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a6e53-e846-46b1-8fea-17839e45900b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:09.530374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a6e53-e846-46b1-8fea-17839e45900b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:09.545840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a6e53-e846-46b1-8fea-17839e45900b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-651a6e53-e846-46b1-8fea-17839e45900b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:09.553477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:09.554382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:09.555670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 33/67] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:03:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:03:09.556548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:09.563707 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a6e53-e846-46b1-8fea-17839e45900b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:09.685524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5597b8f-971a-4695-a244-c3b365825d7c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:09.724312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a6e53-e846-46b1-8fea-17839e45900b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:09.728016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f19396-a020-4b6c-ad93-86fd5190179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:09.729105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f19396-a020-4b6c-ad93-86fd5190179d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:09.729571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f19396-a020-4b6c-ad93-86fd5190179d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:09.730015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f19396-a020-4b6c-ad93-86fd5190179d None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:09.730845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f19396-a020-4b6c-ad93-86fd5190179d None admin] RBAC: Adding query filter params (name=member) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:09.731329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f19396-a020-4b6c-ad93-86fd5190179d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:09.733303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f19396-a020-4b6c-ad93-86fd5190179d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:09.745874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 35/68] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:09.747780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7da7c-c51e-4f29-8764-24c501b8c442 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:09.748267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7da7c-c51e-4f29-8764-24c501b8c442 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:09.748755 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7da7c-c51e-4f29-8764-24c501b8c442 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:09.755756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 36/69] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:09.755756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7da7c-c51e-4f29-8764-24c501b8c442 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:09.770003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392ac43-ee0f-4fd7-9dd8-d307b768ab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:09.771268 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392ac43-ee0f-4fd7-9dd8-d307b768ab6f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:09.771548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392ac43-ee0f-4fd7-9dd8-d307b768ab6f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:09.771857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392ac43-ee0f-4fd7-9dd8-d307b768ab6f None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:09.772588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392ac43-ee0f-4fd7-9dd8-d307b768ab6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:09.772825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392ac43-ee0f-4fd7-9dd8-d307b768ab6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:09.774211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392ac43-ee0f-4fd7-9dd8-d307b768ab6f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:09.777990 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:09.784020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 34/70] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:09.789876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ec35b-8a7d-41c8-aff3-3e6ff808c01b None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:09.790165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ec35b-8a7d-41c8-aff3-3e6ff808c01b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:09.790426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ec35b-8a7d-41c8-aff3-3e6ff808c01b None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:09.791722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 35/71] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:09.793148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e440bf-7946-4d40-be33-cdce5b3687c7 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:09.793457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e440bf-7946-4d40-be33-cdce5b3687c7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:09.793736 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e440bf-7946-4d40-be33-cdce5b3687c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:09.813476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-34e440bf-7946-4d40-be33-cdce5b3687c7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:09.814587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-34e440bf-7946-4d40-be33-cdce5b3687c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:09.868676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7da7c-c51e-4f29-8764-24c501b8c442 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:09.883508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:09.894423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 36/72] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:09.895322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e440bf-7946-4d40-be33-cdce5b3687c7 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:09.923867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653d61d-4369-4ffd-9ff5-54a33e5cdc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:09.924520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653d61d-4369-4ffd-9ff5-54a33e5cdc74 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:09.924791 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653d61d-4369-4ffd-9ff5-54a33e5cdc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:09.925103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653d61d-4369-4ffd-9ff5-54a33e5cdc74 None admin] PATH_INFO: `/v3/users/769b0a726d17470cb59ff5847d0b33a4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:09.946388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653d61d-4369-4ffd-9ff5-54a33e5cdc74 None admin] RBAC: Authorizing `identity:get_user(user_id=769b0a726d17470cb59ff5847d0b33a4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:09.948223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653d61d-4369-4ffd-9ff5-54a33e5cdc74 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:09.950970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 37/73] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:09 2022] GET /identity/v3/users/769b0a726d17470cb59ff5847d0b33a4 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:09.952520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aec98f-a792-4938-9c2c-183e6a424f47 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:09.952803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aec98f-a792-4938-9c2c-183e6a424f47 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:09.953077 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aec98f-a792-4938-9c2c-183e6a424f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:09.978349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-93aec98f-a792-4938-9c2c-183e6a424f47 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:09.979569 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-93aec98f-a792-4938-9c2c-183e6a424f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:10.015207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e440bf-7946-4d40-be33-cdce5b3687c7 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:10.051733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 38/74] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:10.052739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aec98f-a792-4938-9c2c-183e6a424f47 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:10.064019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:10.070178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f45b3a-49cb-4cb3-bd99-67623ac7cf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:10.071350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f45b3a-49cb-4cb3-bd99-67623ac7cf2e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:10.071864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f45b3a-49cb-4cb3-bd99-67623ac7cf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:10.072466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f45b3a-49cb-4cb3-bd99-67623ac7cf2e None admin] PATH_INFO: `/v3/users/f2181a60abf646d28f09241ba111a2d3` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:10.074125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f45b3a-49cb-4cb3-bd99-67623ac7cf2e None admin] RBAC: Authorizing `identity:get_user(user_id=f2181a60abf646d28f09241ba111a2d3)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:10.076333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f45b3a-49cb-4cb3-bd99-67623ac7cf2e None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:10.078582 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 37/75] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:09 2022] GET /identity/v3/users/f2181a60abf646d28f09241ba111a2d3 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:10.080522 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b7210-0f40-4680-a2f3-608284517775 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:10.080970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b7210-0f40-4680-a2f3-608284517775 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:10.081605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b7210-0f40-4680-a2f3-608284517775 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:10.088561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 38/76] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:10.089717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b7210-0f40-4680-a2f3-608284517775 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:10.164502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aec98f-a792-4938-9c2c-183e6a424f47 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:10.169090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:10.201825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598344a-3663-4e55-a56e-d730a8b7e56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:10.202548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598344a-3663-4e55-a56e-d730a8b7e56c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:10.202921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598344a-3663-4e55-a56e-d730a8b7e56c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:10.203206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598344a-3663-4e55-a56e-d730a8b7e56c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:10.213509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b7210-0f40-4680-a2f3-608284517775 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:10.251208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26d613-1cca-4afb-975c-9cf512320c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:10.251848 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26d613-1cca-4afb-975c-9cf512320c31 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:10.252134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26d613-1cca-4afb-975c-9cf512320c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:10.252414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26d613-1cca-4afb-975c-9cf512320c31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:10.277369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598344a-3663-4e55-a56e-d730a8b7e56c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:10.278835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598344a-3663-4e55-a56e-d730a8b7e56c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:10.289474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:10.323522 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26d613-1cca-4afb-975c-9cf512320c31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:10.324941 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26d613-1cca-4afb-975c-9cf512320c31 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:10.379064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 39/77] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:10.380773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b8328-fcf0-4504-beb1-ec5849457898 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:10.381062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b8328-fcf0-4504-beb1-ec5849457898 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:10.381334 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b8328-fcf0-4504-beb1-ec5849457898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:10.387172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:10.398839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-d64b8328-fcf0-4504-beb1-ec5849457898 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:10.400042 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d64b8328-fcf0-4504-beb1-ec5849457898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:10.431500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 39/78] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:10.432944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26d613-1cca-4afb-975c-9cf512320c31 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:10.469336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 40/79] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:10.470169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b8328-fcf0-4504-beb1-ec5849457898 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:10.520122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:10.538701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26d613-1cca-4afb-975c-9cf512320c31 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:10.573225 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd885e-e57b-4519-a4d9-f7b7de15204e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:10.573919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd885e-e57b-4519-a4d9-f7b7de15204e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:10.574154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd885e-e57b-4519-a4d9-f7b7de15204e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:10.574437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd885e-e57b-4519-a4d9-f7b7de15204e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:10.584985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd885e-e57b-4519-a4d9-f7b7de15204e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:10.585843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b8328-fcf0-4504-beb1-ec5849457898 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-b7dd885e-e57b-4519-a4d9-f7b7de15204e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:10.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:10.590716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:10.591472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 40/80] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:10.592153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd885e-e57b-4519-a4d9-f7b7de15204e None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:10.618992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:10.628533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac279e1f-8962-49d2-a837-be9a8095cb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:10.629172 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac279e1f-8962-49d2-a837-be9a8095cb66 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:10.630006 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac279e1f-8962-49d2-a837-be9a8095cb66 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:10.630476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac279e1f-8962-49d2-a837-be9a8095cb66 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:10.640665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac279e1f-8962-49d2-a837-be9a8095cb66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-ac279e1f-8962-49d2-a837-be9a8095cb66 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:10.646101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:10.647008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:10.647967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 41/81] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:03:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:10.648977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac279e1f-8962-49d2-a837-be9a8095cb66 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:10.705240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd885e-e57b-4519-a4d9-f7b7de15204e None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:10.741355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b31adc-3842-4d2c-a960-48eb060fe7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:10.741989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b31adc-3842-4d2c-a960-48eb060fe7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:10.742265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b31adc-3842-4d2c-a960-48eb060fe7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:10.742538 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b31adc-3842-4d2c-a960-48eb060fe7a6 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:10.743648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b31adc-3842-4d2c-a960-48eb060fe7a6 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:10.745136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b31adc-3842-4d2c-a960-48eb060fe7a6 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:10.746752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 41/82] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:10 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:10.746851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:10.747506 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b31adc-3842-4d2c-a960-48eb060fe7a6 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:10.755746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac279e1f-8962-49d2-a837-be9a8095cb66 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:10.789143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989b7ca-247c-481a-acc2-664685d6cb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:10.789769 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989b7ca-247c-481a-acc2-664685d6cb80 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:10.790045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989b7ca-247c-481a-acc2-664685d6cb80 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:10.790324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989b7ca-247c-481a-acc2-664685d6cb80 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:10.791450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989b7ca-247c-481a-acc2-664685d6cb80 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:10.792951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989b7ca-247c-481a-acc2-664685d6cb80 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:10.794889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 42/83] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:10 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:10.799923 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989b7ca-247c-481a-acc2-664685d6cb80 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:10.834392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:10.845724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b31adc-3842-4d2c-a960-48eb060fe7a6 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:10.880094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a72bcd-f3c7-4161-b97c-6380f3379e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:10.880710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a72bcd-f3c7-4161-b97c-6380f3379e19 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:10.880996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a72bcd-f3c7-4161-b97c-6380f3379e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:10.881272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a72bcd-f3c7-4161-b97c-6380f3379e19 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:10.881930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a72bcd-f3c7-4161-b97c-6380f3379e19 None admin] RBAC: Adding query filter params (name=member) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:10.882243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a72bcd-f3c7-4161-b97c-6380f3379e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:10.883665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a72bcd-f3c7-4161-b97c-6380f3379e19 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:10.891611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 42/84] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:10.892419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a72bcd-f3c7-4161-b97c-6380f3379e19 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:10.897348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989b7ca-247c-481a-acc2-664685d6cb80 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:10.931500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c43302-ece0-4799-9439-8cba6d7b5f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:10.932397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c43302-ece0-4799-9439-8cba6d7b5f0a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:10.932682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c43302-ece0-4799-9439-8cba6d7b5f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:10.932964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c43302-ece0-4799-9439-8cba6d7b5f0a None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:10.933629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c43302-ece0-4799-9439-8cba6d7b5f0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:10.933941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c43302-ece0-4799-9439-8cba6d7b5f0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:10.935319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c43302-ece0-4799-9439-8cba6d7b5f0a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:10.944036 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 43/85] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:03:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:10.944821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c43302-ece0-4799-9439-8cba6d7b5f0a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:10.972039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:10.993816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a72bcd-f3c7-4161-b97c-6380f3379e19 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:11.029368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60189d8c-17e8-443d-ab51-c546df0561cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:11.030008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189d8c-17e8-443d-ab51-c546df0561cc None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:11.030294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189d8c-17e8-443d-ab51-c546df0561cc None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:11.030572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60189d8c-17e8-443d-ab51-c546df0561cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:11.031693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60189d8c-17e8-443d-ab51-c546df0561cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:11.033170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60189d8c-17e8-443d-ab51-c546df0561cc None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:11.034738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 43/86] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:11.035497 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60189d8c-17e8-443d-ab51-c546df0561cc None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:11.042844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c43302-ece0-4799-9439-8cba6d7b5f0a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:11.056004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:11.077621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d97efa-8650-4689-b5b2-96c897593009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:11.078231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d97efa-8650-4689-b5b2-96c897593009 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:11.078517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d97efa-8650-4689-b5b2-96c897593009 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:11.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d97efa-8650-4689-b5b2-96c897593009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:11.079529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d97efa-8650-4689-b5b2-96c897593009 None admin] RBAC: Adding query filter params (role.id=f463de99ba174eab9396eef514e34e15, scope.project.id=53b19d145ddf4f3c950098b6269b33e0, user.id=769b0a726d17470cb59ff5847d0b33a4) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:11.079873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d97efa-8650-4689-b5b2-96c897593009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:11.081374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d97efa-8650-4689-b5b2-96c897593009 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:11.086703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 44/87] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:10 2022] GET /identity/v3/role_assignments?user.id=769b0a726d17470cb59ff5847d0b33a4&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=f463de99ba174eab9396eef514e34e15 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:11.087794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d97efa-8650-4689-b5b2-96c897593009 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:11.147859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60189d8c-17e8-443d-ab51-c546df0561cc None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:11.187008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20382a4e-199e-481b-911a-736534a7bc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:11.187850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20382a4e-199e-481b-911a-736534a7bc22 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:11.188137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20382a4e-199e-481b-911a-736534a7bc22 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:11.188424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20382a4e-199e-481b-911a-736534a7bc22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:11.189174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20382a4e-199e-481b-911a-736534a7bc22 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=53b19d145ddf4f3c950098b6269b33e0, user.id=f2181a60abf646d28f09241ba111a2d3) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:11.189686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20382a4e-199e-481b-911a-736534a7bc22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:11.191137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20382a4e-199e-481b-911a-736534a7bc22 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:11.197777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 44/88] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:11 2022] GET /identity/v3/role_assignments?user.id=f2181a60abf646d28f09241ba111a2d3&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:11.198622 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20382a4e-199e-481b-911a-736534a7bc22 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:11.209774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:11.224471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d97efa-8650-4689-b5b2-96c897593009 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:11.279419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47bd65-dfa8-4232-8b62-4e38a086023e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:11.280109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47bd65-dfa8-4232-8b62-4e38a086023e None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:11.280397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47bd65-dfa8-4232-8b62-4e38a086023e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:11.280678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf47bd65-dfa8-4232-8b62-4e38a086023e None admin] PATH_INFO: `/v3/users/769b0a726d17470cb59ff5847d0b33a4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:11.282204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47bd65-dfa8-4232-8b62-4e38a086023e None admin] RBAC: Authorizing `identity:get_user(user_id=769b0a726d17470cb59ff5847d0b33a4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:11.283824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf47bd65-dfa8-4232-8b62-4e38a086023e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:11.285546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 45/89] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:11 2022] GET /identity/v3/users/769b0a726d17470cb59ff5847d0b33a4 => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:11.286751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47bd65-dfa8-4232-8b62-4e38a086023e None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:11.310523 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:11.332291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20382a4e-199e-481b-911a-736534a7bc22 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:11.372090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fc21d-e8ef-4c52-a8ef-c111d0a1b393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:11.372885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fc21d-e8ef-4c52-a8ef-c111d0a1b393 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:11.373171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fc21d-e8ef-4c52-a8ef-c111d0a1b393 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:11.373490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68fc21d-e8ef-4c52-a8ef-c111d0a1b393 None admin] PATH_INFO: `/v3/users/769b0a726d17470cb59ff5847d0b33a4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:11.374787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fc21d-e8ef-4c52-a8ef-c111d0a1b393 None admin] RBAC: Authorizing `identity:get_user(user_id=769b0a726d17470cb59ff5847d0b33a4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:11.376339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68fc21d-e8ef-4c52-a8ef-c111d0a1b393 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:11.378393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 45/90] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:11 2022] GET /identity/v3/users/769b0a726d17470cb59ff5847d0b33a4 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:11.379296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fc21d-e8ef-4c52-a8ef-c111d0a1b393 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:11.424883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf47bd65-dfa8-4232-8b62-4e38a086023e None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:11.469592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37319c-b909-4c9d-bc5c-35340623d4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:11.470528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37319c-b909-4c9d-bc5c-35340623d4c7 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:11.470948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37319c-b909-4c9d-bc5c-35340623d4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:11.471362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37319c-b909-4c9d-bc5c-35340623d4c7 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:11.472743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37319c-b909-4c9d-bc5c-35340623d4c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:11.475366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37319c-b909-4c9d-bc5c-35340623d4c7 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:11.504381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68fc21d-e8ef-4c52-a8ef-c111d0a1b393 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:11.540630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add15ca9-5263-47d3-a604-aa9459b3d7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:11.541469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add15ca9-5263-47d3-a604-aa9459b3d7f9 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:11.541930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add15ca9-5263-47d3-a604-aa9459b3d7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:11.542386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add15ca9-5263-47d3-a604-aa9459b3d7f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:11.563266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:11.599923 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 46/91] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:03:11 2022] POST /identity/v3/users => generated 289 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:03:11.606863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37319c-b909-4c9d-bc5c-35340623d4c7 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:11.644240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:11.644408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add15ca9-5263-47d3-a604-aa9459b3d7f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:11.646440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add15ca9-5263-47d3-a604-aa9459b3d7f9 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:11.814725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37319c-b909-4c9d-bc5c-35340623d4c7 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:11.853798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 46/92] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:11.860788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add15ca9-5263-47d3-a604-aa9459b3d7f9 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:11.880558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1296c-efac-498b-818d-c644133d1bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:11.881635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1296c-efac-498b-818d-c644133d1bdc None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:11.881933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1296c-efac-498b-818d-c644133d1bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:11.882214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd1296c-efac-498b-818d-c644133d1bdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:11.930157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:11.970049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1296c-efac-498b-818d-c644133d1bdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:11.971849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd1296c-efac-498b-818d-c644133d1bdc None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:11.987453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add15ca9-5263-47d3-a604-aa9459b3d7f9 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:12.028221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da764e5e-33cb-43d6-bf57-8e2497d465bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:12.028221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da764e5e-33cb-43d6-bf57-8e2497d465bf None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:12.028221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da764e5e-33cb-43d6-bf57-8e2497d465bf None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:12.028221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da764e5e-33cb-43d6-bf57-8e2497d465bf None admin] PATH_INFO: `/v3/projects/38017532123a473d9cc50c0c5839bd21` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:12.029391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da764e5e-33cb-43d6-bf57-8e2497d465bf None admin] RBAC: Authorizing `identity:get_project(project_id=38017532123a473d9cc50c0c5839bd21)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:12.030852 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da764e5e-33cb-43d6-bf57-8e2497d465bf None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:12.032689 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 47/93] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:11 2022] GET /identity/v3/projects/38017532123a473d9cc50c0c5839bd21 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:12.039110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da764e5e-33cb-43d6-bf57-8e2497d465bf None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:12.042288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:12.149597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 47/94] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:12.153788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da764e5e-33cb-43d6-bf57-8e2497d465bf None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:12.155002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1296c-efac-498b-818d-c644133d1bdc None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:12.170162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:12.190334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be719fd8-67a3-4c16-8e21-9525316f8160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:12.190334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be719fd8-67a3-4c16-8e21-9525316f8160 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:12.190334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be719fd8-67a3-4c16-8e21-9525316f8160 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:12.190334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be719fd8-67a3-4c16-8e21-9525316f8160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:12.190684 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be719fd8-67a3-4c16-8e21-9525316f8160 None admin] RBAC: Adding query filter params (role.id=f463de99ba174eab9396eef514e34e15, scope.project.id=38017532123a473d9cc50c0c5839bd21, user.id=769b0a726d17470cb59ff5847d0b33a4) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:12.190684 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be719fd8-67a3-4c16-8e21-9525316f8160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:12.192870 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be719fd8-67a3-4c16-8e21-9525316f8160 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:12.198585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 48/95] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:12 2022] GET /identity/v3/role_assignments?user.id=769b0a726d17470cb59ff5847d0b33a4&scope.project.id=38017532123a473d9cc50c0c5839bd21&role.id=f463de99ba174eab9396eef514e34e15 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:12.280464 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd1296c-efac-498b-818d-c644133d1bdc None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:12.300219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:12.320292 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bdb66-88f6-4061-9127-7cec9f8540fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:12.320951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bdb66-88f6-4061-9127-7cec9f8540fe None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:12.321238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bdb66-88f6-4061-9127-7cec9f8540fe None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:12.321516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bdb66-88f6-4061-9127-7cec9f8540fe None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:12.323593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bdb66-88f6-4061-9127-7cec9f8540fe None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:12.327845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bdb66-88f6-4061-9127-7cec9f8540fe None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:12.327845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 48/96] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:12 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:12.333787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be719fd8-67a3-4c16-8e21-9525316f8160 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:12.475372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be719fd8-67a3-4c16-8e21-9525316f8160 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:12.532621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d2ff7-2c48-4d49-a890-d9df05304e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:12.533278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d2ff7-2c48-4d49-a890-d9df05304e2b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:12.533544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d2ff7-2c48-4d49-a890-d9df05304e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:12.533819 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d2ff7-2c48-4d49-a890-d9df05304e2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:12.534563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d2ff7-2c48-4d49-a890-d9df05304e2b None admin] RBAC: Adding query filter params (role.id=f8c4be33fa2f4e69a3de71a3c4d7b179, scope.project.id=53b19d145ddf4f3c950098b6269b33e0, user.id=769b0a726d17470cb59ff5847d0b33a4) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:12.534873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d2ff7-2c48-4d49-a890-d9df05304e2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:12.536446 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d2ff7-2c48-4d49-a890-d9df05304e2b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:12.542576 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 49/97] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:12 2022] GET /identity/v3/role_assignments?user.id=769b0a726d17470cb59ff5847d0b33a4&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:13.619641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4a76a-111e-4784-a101-91568f8e98ff None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:13.620187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4a76a-111e-4784-a101-91568f8e98ff None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:13.620369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4a76a-111e-4784-a101-91568f8e98ff None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:13.621627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 49/98] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:13.627633 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9be47f-157b-45d8-86e2-e670ea4cb37e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:13.628181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9be47f-157b-45d8-86e2-e670ea4cb37e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:13.628424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9be47f-157b-45d8-86e2-e670ea4cb37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:13.649220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-2f9be47f-157b-45d8-86e2-e670ea4cb37e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:13.650419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2f9be47f-157b-45d8-86e2-e670ea4cb37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:13.698635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:13.730549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 50/99] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:13.752375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d618-dd7c-47e0-b92b-fcaa9856e82f None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:13.752681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d618-dd7c-47e0-b92b-fcaa9856e82f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:13.752929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d618-dd7c-47e0-b92b-fcaa9856e82f None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:13.754792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 50/100] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:13.760062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e949-b52c-414a-9107-65e3b547e340 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:13.760359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e949-b52c-414a-9107-65e3b547e340 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:13.760638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e949-b52c-414a-9107-65e3b547e340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:13.786627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-4156e949-b52c-414a-9107-65e3b547e340 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:13.787921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4156e949-b52c-414a-9107-65e3b547e340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:13.883151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 51/101] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:14.207078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14d618-dd7c-47e0-b92b-fcaa9856e82f None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:14.319295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c14d618-dd7c-47e0-b92b-fcaa9856e82f None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:14.357007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462564cf-f410-4464-8263-73f7b4388e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:14.357647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462564cf-f410-4464-8263-73f7b4388e99 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:14.357925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462564cf-f410-4464-8263-73f7b4388e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:14.358199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462564cf-f410-4464-8263-73f7b4388e99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:14.373137 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462564cf-f410-4464-8263-73f7b4388e99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-462564cf-f410-4464-8263-73f7b4388e99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:14.378044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:14.378911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:14.380294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 51/102] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:03:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:14.385649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156e949-b52c-414a-9107-65e3b547e340 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:14.460152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462564cf-f410-4464-8263-73f7b4388e99 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:14.526016 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156e949-b52c-414a-9107-65e3b547e340 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:14.528151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:14.540138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:14.627459 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462564cf-f410-4464-8263-73f7b4388e99 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:14.628455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb57324-2947-4627-9a39-7c25f54c4f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:14.629087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb57324-2947-4627-9a39-7c25f54c4f09 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:14.629943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb57324-2947-4627-9a39-7c25f54c4f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:14.630223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb57324-2947-4627-9a39-7c25f54c4f09 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:14.630905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb57324-2947-4627-9a39-7c25f54c4f09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:14.631211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb57324-2947-4627-9a39-7c25f54c4f09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:14.635793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb57324-2947-4627-9a39-7c25f54c4f09 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:14.650041 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 52/103] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:14.655259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb57324-2947-4627-9a39-7c25f54c4f09 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:14.696302 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de7f34-90ee-4f9d-9a91-c6c5404553cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:14.697290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de7f34-90ee-4f9d-9a91-c6c5404553cb None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:14.697790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de7f34-90ee-4f9d-9a91-c6c5404553cb None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:14.698273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de7f34-90ee-4f9d-9a91-c6c5404553cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:14.709447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de7f34-90ee-4f9d-9a91-c6c5404553cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-83de7f34-90ee-4f9d-9a91-c6c5404553cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:14.715556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:14.716455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:14.717204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:14.719643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 52/104] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:14.729031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de7f34-90ee-4f9d-9a91-c6c5404553cb None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:14.841672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb57324-2947-4627-9a39-7c25f54c4f09 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:14.881314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb081c2c-e84b-4f22-9ff9-c9715271feff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:14.881948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081c2c-e84b-4f22-9ff9-c9715271feff None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:14.882228 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081c2c-e84b-4f22-9ff9-c9715271feff None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:14.882503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081c2c-e84b-4f22-9ff9-c9715271feff None admin] PATH_INFO: `/v3/users/f2181a60abf646d28f09241ba111a2d3` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:14.883631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb081c2c-e84b-4f22-9ff9-c9715271feff None admin] RBAC: Authorizing `identity:get_user(user_id=f2181a60abf646d28f09241ba111a2d3)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:14.885739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb081c2c-e84b-4f22-9ff9-c9715271feff None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:14.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 53/105] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:14 2022] GET /identity/v3/users/f2181a60abf646d28f09241ba111a2d3 => generated 247 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:14.893904 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb081c2c-e84b-4f22-9ff9-c9715271feff None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:14.941646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de7f34-90ee-4f9d-9a91-c6c5404553cb None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:14.945565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:14.968213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:15.031484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c20d7-7aeb-46a4-8e91-4e878407e065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:15.032132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c20d7-7aeb-46a4-8e91-4e878407e065 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.032414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c20d7-7aeb-46a4-8e91-4e878407e065 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.032686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c20d7-7aeb-46a4-8e91-4e878407e065 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.033419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c20d7-7aeb-46a4-8e91-4e878407e065 None admin] RBAC: Adding query filter params (name=member) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:15.033725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c20d7-7aeb-46a4-8e91-4e878407e065 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:15.035128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c20d7-7aeb-46a4-8e91-4e878407e065 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:15.059893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 53/106] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:15.068142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c20d7-7aeb-46a4-8e91-4e878407e065 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:15.129112 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb081c2c-e84b-4f22-9ff9-c9715271feff None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:15.238769 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c20d7-7aeb-46a4-8e91-4e878407e065 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:15.239266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf08c2-768e-488c-946a-7a0923ea74bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:15.239953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf08c2-768e-488c-946a-7a0923ea74bc None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.240234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf08c2-768e-488c-946a-7a0923ea74bc None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.240507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adf08c2-768e-488c-946a-7a0923ea74bc None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.244462 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf08c2-768e-488c-946a-7a0923ea74bc None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:15.245939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adf08c2-768e-488c-946a-7a0923ea74bc None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:15.248068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 54/107] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:14 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:15.253668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf08c2-768e-488c-946a-7a0923ea74bc None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:15.294059 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad95f73-560a-4b39-8a0a-0e394d8d2368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:15.295154 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad95f73-560a-4b39-8a0a-0e394d8d2368 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.295442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad95f73-560a-4b39-8a0a-0e394d8d2368 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.295722 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad95f73-560a-4b39-8a0a-0e394d8d2368 None admin] PATH_INFO: `/v3/users/769b0a726d17470cb59ff5847d0b33a4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.300496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad95f73-560a-4b39-8a0a-0e394d8d2368 None admin] RBAC: Authorizing `identity:get_user(user_id=769b0a726d17470cb59ff5847d0b33a4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:15.302400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad95f73-560a-4b39-8a0a-0e394d8d2368 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:15.304439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 54/108] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:15 2022] GET /identity/v3/users/769b0a726d17470cb59ff5847d0b33a4 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:15.316613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad95f73-560a-4b39-8a0a-0e394d8d2368 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:15.380226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:15.469731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adf08c2-768e-488c-946a-7a0923ea74bc None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:15.473957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:15.481018 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad95f73-560a-4b39-8a0a-0e394d8d2368 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:15.515341 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f8307-f4c4-455a-a5c2-910f42175579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:15.515949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f8307-f4c4-455a-a5c2-910f42175579 None admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.516227 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f8307-f4c4-455a-a5c2-910f42175579 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.516502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f8307-f4c4-455a-a5c2-910f42175579 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0/users/f2181a60abf646d28f09241ba111a2d3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.518489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f8307-f4c4-455a-a5c2-910f42175579 None admin] RBAC: Authorizing `identity:create_grant(project_id=53b19d145ddf4f3c950098b6269b33e0, user_id=f2181a60abf646d28f09241ba111a2d3, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:15.520138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f8307-f4c4-455a-a5c2-910f42175579 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:15.522945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f7d8c-5a11-423a-8fdd-fbe8ae755611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:15.523583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f7d8c-5a11-423a-8fdd-fbe8ae755611 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.523906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f7d8c-5a11-423a-8fdd-fbe8ae755611 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.524311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f7d8c-5a11-423a-8fdd-fbe8ae755611 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.526093 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f7d8c-5a11-423a-8fdd-fbe8ae755611 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:15.527669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f7d8c-5a11-423a-8fdd-fbe8ae755611 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:15.529495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 55/109] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:15 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:15.530425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e6723-8b41-4240-a674-53821ab2dabe None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.530709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e6723-8b41-4240-a674-53821ab2dabe None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.530983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e6723-8b41-4240-a674-53821ab2dabe None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.532308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 56/110] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:15.533318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53d4d0-3003-4277-9d6f-505c4fed4494 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.533599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53d4d0-3003-4277-9d6f-505c4fed4494 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.533875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53d4d0-3003-4277-9d6f-505c4fed4494 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.535121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 57/111] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:15.536725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89163e64-a515-4669-a07e-7e5f8f7ab44c None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.537050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89163e64-a515-4669-a07e-7e5f8f7ab44c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.537332 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89163e64-a515-4669-a07e-7e5f8f7ab44c None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.538662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 58/112] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:15.539281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89163e64-a515-4669-a07e-7e5f8f7ab44c None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:15.588827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 55/113] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:03:15 2022] PUT /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0/users/f2181a60abf646d28f09241ba111a2d3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:03:15.597237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebf6c5-c869-4691-9a22-51506faf446e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.597528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebf6c5-c869-4691-9a22-51506faf446e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.597806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebf6c5-c869-4691-9a22-51506faf446e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.637737 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-62ebf6c5-c869-4691-9a22-51506faf446e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:15.640498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-62ebf6c5-c869-4691-9a22-51506faf446e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:15.653298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89163e64-a515-4669-a07e-7e5f8f7ab44c None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:15.655450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:15.704225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427952a-d8bd-4326-bba2-f050389da817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:15.705637 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427952a-d8bd-4326-bba2-f050389da817 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.705947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427952a-d8bd-4326-bba2-f050389da817 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.706235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427952a-d8bd-4326-bba2-f050389da817 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0/users/769b0a726d17470cb59ff5847d0b33a4/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.708519 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427952a-d8bd-4326-bba2-f050389da817 None admin] RBAC: Authorizing `identity:create_grant(project_id=53b19d145ddf4f3c950098b6269b33e0, user_id=769b0a726d17470cb59ff5847d0b33a4, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:15.710800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427952a-d8bd-4326-bba2-f050389da817 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:15.741259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 56/114] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:15.742890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd39ba-3510-4342-8c16-a5286a4d409b None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.743379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd39ba-3510-4342-8c16-a5286a4d409b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.743896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd39ba-3510-4342-8c16-a5286a4d409b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.764778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-24dd39ba-3510-4342-8c16-a5286a4d409b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:15.779935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-24dd39ba-3510-4342-8c16-a5286a4d409b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:15.799369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 59/115] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:03:15 2022] PUT /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0/users/769b0a726d17470cb59ff5847d0b33a4/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:15.800795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73cbdd-420c-401c-99eb-afb5f1a64acf None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.801121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73cbdd-420c-401c-99eb-afb5f1a64acf None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.801399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73cbdd-420c-401c-99eb-afb5f1a64acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.828355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-cd73cbdd-420c-401c-99eb-afb5f1a64acf None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:15.829589 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-cd73cbdd-420c-401c-99eb-afb5f1a64acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:15.887503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 57/116] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:15.888799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74dfc9-a17a-4e2c-88cb-6a175cfaa6a8 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.889093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74dfc9-a17a-4e2c-88cb-6a175cfaa6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.889373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74dfc9-a17a-4e2c-88cb-6a175cfaa6a8 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.891026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 58/117] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:15.897143 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b58be-e138-40a7-bf5b-5e7ee3d0210d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:15.897436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b58be-e138-40a7-bf5b-5e7ee3d0210d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:15.897721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b58be-e138-40a7-bf5b-5e7ee3d0210d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:15.919482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-092b58be-e138-40a7-bf5b-5e7ee3d0210d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:15.920797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-092b58be-e138-40a7-bf5b-5e7ee3d0210d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:15.952623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:15.961618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 60/118] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:16.103060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 59/119] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:16.137657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73cbdd-420c-401c-99eb-afb5f1a64acf None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:16.228193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b58be-e138-40a7-bf5b-5e7ee3d0210d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:16.251541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:16.262294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73cbdd-420c-401c-99eb-afb5f1a64acf None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:16.302843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d8462-dc8d-42d1-bb57-f7dd958de709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:16.303556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d8462-dc8d-42d1-bb57-f7dd958de709 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:16.303921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d8462-dc8d-42d1-bb57-f7dd958de709 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:16.304256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d8462-dc8d-42d1-bb57-f7dd958de709 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:16.315511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d8462-dc8d-42d1-bb57-f7dd958de709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-3c9d8462-dc8d-42d1-bb57-f7dd958de709 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:16.321669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:16.322532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:16.323678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 61/120] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:16.329020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d8462-dc8d-42d1-bb57-f7dd958de709 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:16.432280 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b58be-e138-40a7-bf5b-5e7ee3d0210d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:16.449307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d8462-dc8d-42d1-bb57-f7dd958de709 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:16.488105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4923-38e6-4edf-a507-ad6985363512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:16.489322 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4923-38e6-4edf-a507-ad6985363512 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:16.489608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4923-38e6-4edf-a507-ad6985363512 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:16.489888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e4923-38e6-4edf-a507-ad6985363512 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:16.490605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e4923-38e6-4edf-a507-ad6985363512 None admin] RBAC: Adding query filter params (name=member) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:16.490913 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e4923-38e6-4edf-a507-ad6985363512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:16.492394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e4923-38e6-4edf-a507-ad6985363512 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:16.502742 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 62/121] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:16.503374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4923-38e6-4edf-a507-ad6985363512 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:16.519721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae2a7f-ef5d-47f2-81be-928a4044703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:16.520275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae2a7f-ef5d-47f2-81be-928a4044703d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:16.520559 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae2a7f-ef5d-47f2-81be-928a4044703d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:16.520837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae2a7f-ef5d-47f2-81be-928a4044703d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:16.532634 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:16.539175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae2a7f-ef5d-47f2-81be-928a4044703d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-37ae2a7f-ef5d-47f2-81be-928a4044703d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:16.544182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:16.550299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 60/122] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:03:16.551616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae2a7f-ef5d-47f2-81be-928a4044703d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:16.572955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:16.660114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae2a7f-ef5d-47f2-81be-928a4044703d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:16.675225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e4923-38e6-4edf-a507-ad6985363512 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:16.701038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407efe37-96a7-44eb-b814-1815b38de852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:16.701598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407efe37-96a7-44eb-b814-1815b38de852 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:16.701879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407efe37-96a7-44eb-b814-1815b38de852 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:16.702156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407efe37-96a7-44eb-b814-1815b38de852 None admin] PATH_INFO: `/v3/users/769b0a726d17470cb59ff5847d0b33a4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:16.703309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407efe37-96a7-44eb-b814-1815b38de852 None admin] RBAC: Authorizing `identity:get_user(user_id=769b0a726d17470cb59ff5847d0b33a4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:16.704808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407efe37-96a7-44eb-b814-1815b38de852 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:16.706396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 61/123] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:16 2022] GET /identity/v3/users/769b0a726d17470cb59ff5847d0b33a4 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:16.711085 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407efe37-96a7-44eb-b814-1815b38de852 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:16.728286 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea109d4-03c1-4f04-a7fd-2a538ea2f707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:16.728932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea109d4-03c1-4f04-a7fd-2a538ea2f707 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:16.729212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea109d4-03c1-4f04-a7fd-2a538ea2f707 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:16.729488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea109d4-03c1-4f04-a7fd-2a538ea2f707 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:16.730665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea109d4-03c1-4f04-a7fd-2a538ea2f707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:16.732416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea109d4-03c1-4f04-a7fd-2a538ea2f707 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:16.734899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 63/124] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:16.735926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea109d4-03c1-4f04-a7fd-2a538ea2f707 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:16.806026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:16.825575 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407efe37-96a7-44eb-b814-1815b38de852 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:16.848389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:16.861423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643e4a0-cbac-48ed-83ab-1fb1e4d1f577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:16.862092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643e4a0-cbac-48ed-83ab-1fb1e4d1f577 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:16.862379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643e4a0-cbac-48ed-83ab-1fb1e4d1f577 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:16.862648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7643e4a0-cbac-48ed-83ab-1fb1e4d1f577 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:16.863310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643e4a0-cbac-48ed-83ab-1fb1e4d1f577 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:16.864290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643e4a0-cbac-48ed-83ab-1fb1e4d1f577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:16.865923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7643e4a0-cbac-48ed-83ab-1fb1e4d1f577 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:16.872821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea109d4-03c1-4f04-a7fd-2a538ea2f707 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:16.874273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 62/125] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:16.875062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643e4a0-cbac-48ed-83ab-1fb1e4d1f577 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:16.917453 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec60d8-2b01-4e79-87f2-2484e16770bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:16.918387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec60d8-2b01-4e79-87f2-2484e16770bb None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:16.919236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec60d8-2b01-4e79-87f2-2484e16770bb None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:16.919694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec60d8-2b01-4e79-87f2-2484e16770bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:16.936859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec60d8-2b01-4e79-87f2-2484e16770bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-eaec60d8-2b01-4e79-87f2-2484e16770bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:16.944758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:16.945652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:16.946518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:16.954062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 64/126] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:03:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:16.955114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec60d8-2b01-4e79-87f2-2484e16770bb None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:16.989213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7643e4a0-cbac-48ed-83ab-1fb1e4d1f577 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:17.026074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5cccf-8725-4d8b-b3de-ca1840b04a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:17.026914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5cccf-8725-4d8b-b3de-ca1840b04a4f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:17.027294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5cccf-8725-4d8b-b3de-ca1840b04a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:17.027670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5cccf-8725-4d8b-b3de-ca1840b04a4f None admin] PATH_INFO: `/v3/projects/38017532123a473d9cc50c0c5839bd21` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:17.028994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5cccf-8725-4d8b-b3de-ca1840b04a4f None admin] RBAC: Authorizing `identity:get_project(project_id=38017532123a473d9cc50c0c5839bd21)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:17.030694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5cccf-8725-4d8b-b3de-ca1840b04a4f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:17.037810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 63/127] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:16 2022] GET /identity/v3/projects/38017532123a473d9cc50c0c5839bd21 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:17.037810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5cccf-8725-4d8b-b3de-ca1840b04a4f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:17.053877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:17.104792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec60d8-2b01-4e79-87f2-2484e16770bb None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:17.143358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3de657-d5d1-4b75-8df2-9b42bebe0d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:17.144463 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3de657-d5d1-4b75-8df2-9b42bebe0d2a None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:17.144894 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3de657-d5d1-4b75-8df2-9b42bebe0d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:17.145541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3de657-d5d1-4b75-8df2-9b42bebe0d2a None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:17.146300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3de657-d5d1-4b75-8df2-9b42bebe0d2a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:17.148420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3de657-d5d1-4b75-8df2-9b42bebe0d2a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:17.167600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:17.170248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5cccf-8725-4d8b-b3de-ca1840b04a4f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:17.211879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb93f28-7043-462c-b2dd-019e90764cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:17.212764 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb93f28-7043-462c-b2dd-019e90764cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:17.213189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb93f28-7043-462c-b2dd-019e90764cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:17.213608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb93f28-7043-462c-b2dd-019e90764cf7 None admin] PATH_INFO: `/v3/users/1cb85440fc1a4f8a9646580b3e466d6d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:17.232969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb93f28-7043-462c-b2dd-019e90764cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=1cb85440fc1a4f8a9646580b3e466d6d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:17.235743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb93f28-7043-462c-b2dd-019e90764cf7 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:17.237816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 64/128] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:17 2022] GET /identity/v3/users/1cb85440fc1a4f8a9646580b3e466d6d => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:17.240719 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb93f28-7043-462c-b2dd-019e90764cf7 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:17.245757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 65/129] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:03:16 2022] POST /identity/v3/projects => generated 302 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:03:17.246869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3de657-d5d1-4b75-8df2-9b42bebe0d2a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:17.373199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3de657-d5d1-4b75-8df2-9b42bebe0d2a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:17.382579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:17.417596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77adf5c-b9c9-4cef-8a29-4580a29b6f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:17.418525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77adf5c-b9c9-4cef-8a29-4580a29b6f1d None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:17.418951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77adf5c-b9c9-4cef-8a29-4580a29b6f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:17.419372 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77adf5c-b9c9-4cef-8a29-4580a29b6f1d None admin] PATH_INFO: `/v3/projects/38017532123a473d9cc50c0c5839bd21/users/769b0a726d17470cb59ff5847d0b33a4/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:17.421814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77adf5c-b9c9-4cef-8a29-4580a29b6f1d None admin] RBAC: Authorizing `identity:create_grant(project_id=38017532123a473d9cc50c0c5839bd21, user_id=769b0a726d17470cb59ff5847d0b33a4, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:17.423629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77adf5c-b9c9-4cef-8a29-4580a29b6f1d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:17.453473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb93f28-7043-462c-b2dd-019e90764cf7 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:17.499243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04133a94-5ea3-4074-96fb-2cfd989fd333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:17.500206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04133a94-5ea3-4074-96fb-2cfd989fd333 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:17.500705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04133a94-5ea3-4074-96fb-2cfd989fd333 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:17.501229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04133a94-5ea3-4074-96fb-2cfd989fd333 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:17.502608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04133a94-5ea3-4074-96fb-2cfd989fd333 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:17.505051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04133a94-5ea3-4074-96fb-2cfd989fd333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:17.505051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04133a94-5ea3-4074-96fb-2cfd989fd333 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:17.509636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 66/130] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:03:17 2022] PUT /identity/v3/projects/38017532123a473d9cc50c0c5839bd21/users/769b0a726d17470cb59ff5847d0b33a4/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:17.516564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 65/131] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:03:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:17.516564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04133a94-5ea3-4074-96fb-2cfd989fd333 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:17.523646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77adf5c-b9c9-4cef-8a29-4580a29b6f1d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:17.632211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:17.651844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04133a94-5ea3-4074-96fb-2cfd989fd333 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:17.688833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:17.695833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821203ff-bacd-44ea-bb53-42d1d44d1c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:17.696526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821203ff-bacd-44ea-bb53-42d1d44d1c57 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:17.696815 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821203ff-bacd-44ea-bb53-42d1d44d1c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:17.697089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821203ff-bacd-44ea-bb53-42d1d44d1c57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:17.699790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77adf5c-b9c9-4cef-8a29-4580a29b6f1d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:17.772885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d0196-2000-434d-8169-7723066283a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:17.773576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d0196-2000-434d-8169-7723066283a5 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:17.773932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d0196-2000-434d-8169-7723066283a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:17.774284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d0196-2000-434d-8169-7723066283a5 None admin] PATH_INFO: `/v3/users/769b0a726d17470cb59ff5847d0b33a4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:17.775889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d0196-2000-434d-8169-7723066283a5 None admin] RBAC: Authorizing `identity:get_user(user_id=769b0a726d17470cb59ff5847d0b33a4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:17.777513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d0196-2000-434d-8169-7723066283a5 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:17.783508 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 67/132] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:17 2022] GET /identity/v3/users/769b0a726d17470cb59ff5847d0b33a4 => generated 275 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:17.786903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d0196-2000-434d-8169-7723066283a5 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:17.833586 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821203ff-bacd-44ea-bb53-42d1d44d1c57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:17.835327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821203ff-bacd-44ea-bb53-42d1d44d1c57 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:17.951798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d0196-2000-434d-8169-7723066283a5 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:17.994112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:18.022417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473620ae-17c0-4596-8f11-a8ba53e1a0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:18.023138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473620ae-17c0-4596-8f11-a8ba53e1a0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:18.023363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473620ae-17c0-4596-8f11-a8ba53e1a0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:18.023642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473620ae-17c0-4596-8f11-a8ba53e1a0d5 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:18.026031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473620ae-17c0-4596-8f11-a8ba53e1a0d5 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:18.027512 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473620ae-17c0-4596-8f11-a8ba53e1a0d5 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:18.029189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 68/133] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:17 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:18.035003 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473620ae-17c0-4596-8f11-a8ba53e1a0d5 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:18.078768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 66/134] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:18.084708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821203ff-bacd-44ea-bb53-42d1d44d1c57 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:18.143731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:18.153710 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473620ae-17c0-4596-8f11-a8ba53e1a0d5 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:18.188485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45136834-bc1a-451a-be79-1d15f31ecd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:18.189151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45136834-bc1a-451a-be79-1d15f31ecd95 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:18.189490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45136834-bc1a-451a-be79-1d15f31ecd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:18.189765 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45136834-bc1a-451a-be79-1d15f31ecd95 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0/users/769b0a726d17470cb59ff5847d0b33a4/roles/f8c4be33fa2f4e69a3de71a3c4d7b179` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:18.191974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45136834-bc1a-451a-be79-1d15f31ecd95 None admin] RBAC: Authorizing `identity:create_grant(project_id=53b19d145ddf4f3c950098b6269b33e0, user_id=769b0a726d17470cb59ff5847d0b33a4, role_id=f8c4be33fa2f4e69a3de71a3c4d7b179)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:18.193442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45136834-bc1a-451a-be79-1d15f31ecd95 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:18.248075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821203ff-bacd-44ea-bb53-42d1d44d1c57 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:18.272652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 69/135] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:03:18 2022] PUT /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0/users/769b0a726d17470cb59ff5847d0b33a4/roles/f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:18.310081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e48168-6699-4280-b074-95542f563977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:18.310652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e48168-6699-4280-b074-95542f563977 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:18.310928 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e48168-6699-4280-b074-95542f563977 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:18.311204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e48168-6699-4280-b074-95542f563977 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:18.312458 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e48168-6699-4280-b074-95542f563977 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:18.313938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e48168-6699-4280-b074-95542f563977 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:18.315827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 67/136] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:18 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:18.324260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45136834-bc1a-451a-be79-1d15f31ecd95 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:18.386008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:18.449923 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45136834-bc1a-451a-be79-1d15f31ecd95 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:18.494450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c5c44-34b5-4040-a3e2-bd6a41acb07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:18.495074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c5c44-34b5-4040-a3e2-bd6a41acb07d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:18.495357 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c5c44-34b5-4040-a3e2-bd6a41acb07d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:18.495633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c5c44-34b5-4040-a3e2-bd6a41acb07d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:18.496555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c5c44-34b5-4040-a3e2-bd6a41acb07d None admin] RBAC: Adding query filter params (role.id=4f0b83503c8847079d53d3dd089d98cb, scope.project.id=53b19d145ddf4f3c950098b6269b33e0, user.id=1cb85440fc1a4f8a9646580b3e466d6d) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:18.496868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c5c44-34b5-4040-a3e2-bd6a41acb07d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:18.498445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c5c44-34b5-4040-a3e2-bd6a41acb07d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:18.508858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 70/137] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:18 2022] GET /identity/v3/role_assignments?user.id=1cb85440fc1a4f8a9646580b3e466d6d&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=4f0b83503c8847079d53d3dd089d98cb => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:18.578203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f901a-7c1f-4572-bc1f-f4698a9e2f36 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:18.578384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f901a-7c1f-4572-bc1f-f4698a9e2f36 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:18.578660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f901a-7c1f-4572-bc1f-f4698a9e2f36 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:18.580090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 68/138] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:18.591282 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea575e-f695-41d3-9597-85de3962b829 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:18.591587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea575e-f695-41d3-9597-85de3962b829 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:18.591861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea575e-f695-41d3-9597-85de3962b829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:18.616849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-f6ea575e-f695-41d3-9597-85de3962b829 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:18.618466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f6ea575e-f695-41d3-9597-85de3962b829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:18.729850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2186b-e1dc-46d0-880a-f749873617a1 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:18.730356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2186b-e1dc-46d0-880a-f749873617a1 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:18.730428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2186b-e1dc-46d0-880a-f749873617a1 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:18.733496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 69/139] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:18.738010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70979f37-a54d-4d6d-9b4e-19b89202bf80 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:18.738264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 71/140] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:18.739231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70979f37-a54d-4d6d-9b4e-19b89202bf80 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:18.739403 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70979f37-a54d-4d6d-9b4e-19b89202bf80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:18.769538 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-70979f37-a54d-4d6d-9b4e-19b89202bf80 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:18.770788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-70979f37-a54d-4d6d-9b4e-19b89202bf80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:18.872959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 70/141] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:19.288507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea575e-f695-41d3-9597-85de3962b829 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:19.289023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70979f37-a54d-4d6d-9b4e-19b89202bf80 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:19.321010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:19.353184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:19.397987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70979f37-a54d-4d6d-9b4e-19b89202bf80 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:19.436020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b987f0a-cc02-44db-802d-d79526698d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:19.437101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b987f0a-cc02-44db-802d-d79526698d9a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:19.437572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b987f0a-cc02-44db-802d-d79526698d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:19.438061 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b987f0a-cc02-44db-802d-d79526698d9a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:19.442824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea575e-f695-41d3-9597-85de3962b829 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:19.449347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b987f0a-cc02-44db-802d-d79526698d9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-1b987f0a-cc02-44db-802d-d79526698d9a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:19.454405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:19.455337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:19.456114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:19.458327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 71/142] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:19.464098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b987f0a-cc02-44db-802d-d79526698d9a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:19.505501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf9478-6fc3-4002-a29b-820e6a2981ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:19.506163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf9478-6fc3-4002-a29b-820e6a2981ec None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:19.506450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf9478-6fc3-4002-a29b-820e6a2981ec None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:19.506730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf9478-6fc3-4002-a29b-820e6a2981ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:19.529138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faf9478-6fc3-4002-a29b-820e6a2981ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-8faf9478-6fc3-4002-a29b-820e6a2981ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:19.534051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:19.534896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:19.536267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 72/143] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:03:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:19.543354 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf9478-6fc3-4002-a29b-820e6a2981ec None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:19.572156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:19.580336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b987f0a-cc02-44db-802d-d79526698d9a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:19.638783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6b64-803e-4318-9ebf-ac2b07a986d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:19.639800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6b64-803e-4318-9ebf-ac2b07a986d7 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:19.640271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6b64-803e-4318-9ebf-ac2b07a986d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:19.640731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6b64-803e-4318-9ebf-ac2b07a986d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:19.641598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6b64-803e-4318-9ebf-ac2b07a986d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:19.642105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6b64-803e-4318-9ebf-ac2b07a986d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:19.643500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6b64-803e-4318-9ebf-ac2b07a986d7 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:19.663556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 72/144] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:19.666767 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6b64-803e-4318-9ebf-ac2b07a986d7 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:19.758448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf9478-6fc3-4002-a29b-820e6a2981ec None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:19.764649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:19.824669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6b64-803e-4318-9ebf-ac2b07a986d7 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:19.839168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba31f9-4ec4-458a-b0b0-935fe1358cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:19.840225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba31f9-4ec4-458a-b0b0-935fe1358cfa None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:19.840749 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba31f9-4ec4-458a-b0b0-935fe1358cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:19.841219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba31f9-4ec4-458a-b0b0-935fe1358cfa None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:19.842123 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba31f9-4ec4-458a-b0b0-935fe1358cfa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:19.842713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba31f9-4ec4-458a-b0b0-935fe1358cfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:19.844452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba31f9-4ec4-458a-b0b0-935fe1358cfa None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:19.854394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 73/145] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:19.860986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba31f9-4ec4-458a-b0b0-935fe1358cfa None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:19.889068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231953f-cb7a-4776-ba0b-b0dff6f4bf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:19.889713 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231953f-cb7a-4776-ba0b-b0dff6f4bf4b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:19.889992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231953f-cb7a-4776-ba0b-b0dff6f4bf4b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:19.890272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231953f-cb7a-4776-ba0b-b0dff6f4bf4b None admin] PATH_INFO: `/v3/users/769b0a726d17470cb59ff5847d0b33a4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:19.893730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231953f-cb7a-4776-ba0b-b0dff6f4bf4b None admin] RBAC: Authorizing `identity:get_user(user_id=769b0a726d17470cb59ff5847d0b33a4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:19.895801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231953f-cb7a-4776-ba0b-b0dff6f4bf4b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:19.897765 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 73/146] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:19 2022] GET /identity/v3/users/769b0a726d17470cb59ff5847d0b33a4 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:19.903091 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231953f-cb7a-4776-ba0b-b0dff6f4bf4b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:19.940736 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:20.049038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba31f9-4ec4-458a-b0b0-935fe1358cfa None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:20.098234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231953f-cb7a-4776-ba0b-b0dff6f4bf4b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:20.114718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a51651-bb7a-4d67-8002-6e3087fbeadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:20.116418 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a51651-bb7a-4d67-8002-6e3087fbeadc None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:20.117608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a51651-bb7a-4d67-8002-6e3087fbeadc None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:20.118860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a51651-bb7a-4d67-8002-6e3087fbeadc None admin] PATH_INFO: `/v3/users/f2181a60abf646d28f09241ba111a2d3` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:20.120387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a51651-bb7a-4d67-8002-6e3087fbeadc None admin] RBAC: Authorizing `identity:get_user(user_id=f2181a60abf646d28f09241ba111a2d3)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:20.124096 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a51651-bb7a-4d67-8002-6e3087fbeadc None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:20.128451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 74/147] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:19 2022] GET /identity/v3/users/f2181a60abf646d28f09241ba111a2d3 => generated 247 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:20.134474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a51651-bb7a-4d67-8002-6e3087fbeadc None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:20.155366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37ad53-e74d-44ac-89e7-9d1f827b633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:20.157092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37ad53-e74d-44ac-89e7-9d1f827b633b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:20.157770 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37ad53-e74d-44ac-89e7-9d1f827b633b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:20.159296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37ad53-e74d-44ac-89e7-9d1f827b633b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:20.198241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:20.274200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37ad53-e74d-44ac-89e7-9d1f827b633b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:20.276194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e37ad53-e74d-44ac-89e7-9d1f827b633b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:20.320976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:20.325237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a51651-bb7a-4d67-8002-6e3087fbeadc None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:20.377862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d46b29-1e2c-43f3-82d3-734cb6056791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:20.378421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d46b29-1e2c-43f3-82d3-734cb6056791 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:20.378699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d46b29-1e2c-43f3-82d3-734cb6056791 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:20.383786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d46b29-1e2c-43f3-82d3-734cb6056791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:20.431370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 74/148] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 16:03:20.437361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37ad53-e74d-44ac-89e7-9d1f827b633b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:20.492294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d46b29-1e2c-43f3-82d3-734cb6056791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:20.493783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d46b29-1e2c-43f3-82d3-734cb6056791 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:20.530590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:20.547540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e37ad53-e74d-44ac-89e7-9d1f827b633b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:20.570925 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:20.586272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69894085-87d6-4fd5-93d6-7b57250c6b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:20.587194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69894085-87d6-4fd5-93d6-7b57250c6b8d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:20.587631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69894085-87d6-4fd5-93d6-7b57250c6b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:20.588101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69894085-87d6-4fd5-93d6-7b57250c6b8d None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:20.589437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69894085-87d6-4fd5-93d6-7b57250c6b8d None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:20.591095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69894085-87d6-4fd5-93d6-7b57250c6b8d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:20.593038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 75/149] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:20 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:20.599640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69894085-87d6-4fd5-93d6-7b57250c6b8d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:20.603171 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 75/150] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 16:03:20.608746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d46b29-1e2c-43f3-82d3-734cb6056791 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:20.731018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69894085-87d6-4fd5-93d6-7b57250c6b8d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:20.739680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d46b29-1e2c-43f3-82d3-734cb6056791 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:20.775746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e4dba-9256-4d46-8ef5-d3a474c84ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:20.776532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e4dba-9256-4d46-8ef5-d3a474c84ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:20.776906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e4dba-9256-4d46-8ef5-d3a474c84ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:20.777321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e4dba-9256-4d46-8ef5-d3a474c84ed5 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:20.778340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc2971-e525-4fe0-8a71-4aeeb5324fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:20.778635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e4dba-9256-4d46-8ef5-d3a474c84ed5 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:20.779254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc2971-e525-4fe0-8a71-4aeeb5324fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:20.779672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc2971-e525-4fe0-8a71-4aeeb5324fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:20.780120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc2971-e525-4fe0-8a71-4aeeb5324fa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:20.780482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e4dba-9256-4d46-8ef5-d3a474c84ed5 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:20.781143 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc2971-e525-4fe0-8a71-4aeeb5324fa2 None admin] RBAC: Adding query filter params (role.id=f463de99ba174eab9396eef514e34e15, scope.project.id=53b19d145ddf4f3c950098b6269b33e0, user.id=769b0a726d17470cb59ff5847d0b33a4) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:20.781590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc2971-e525-4fe0-8a71-4aeeb5324fa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:20.782843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 76/151] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:20 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:20.783331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fc2971-e525-4fe0-8a71-4aeeb5324fa2 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:20.784210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960c532-22de-4308-bef9-4cefb1a08d81 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:20.784580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960c532-22de-4308-bef9-4cefb1a08d81 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:20.784960 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960c532-22de-4308-bef9-4cefb1a08d81 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:20.787396 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 77/152] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:20.791145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 76/153] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:20 2022] GET /identity/v3/role_assignments?user.id=769b0a726d17470cb59ff5847d0b33a4&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=f463de99ba174eab9396eef514e34e15 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:20.795803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647506a6-7fd3-445b-9c75-de1810a6f34b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:20.795803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647506a6-7fd3-445b-9c75-de1810a6f34b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:20.795803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647506a6-7fd3-445b-9c75-de1810a6f34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:20.799039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc2971-e525-4fe0-8a71-4aeeb5324fa2 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:20.804104 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:20.828401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-647506a6-7fd3-445b-9c75-de1810a6f34b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:20.829632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-647506a6-7fd3-445b-9c75-de1810a6f34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:20.891876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:20.984236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 78/154] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:20.987382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc2971-e525-4fe0-8a71-4aeeb5324fa2 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:21.050440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca0832-ec2f-4cb0-881a-1cb110559af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:21.051432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca0832-ec2f-4cb0-881a-1cb110559af1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:21.052047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca0832-ec2f-4cb0-881a-1cb110559af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:21.052522 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca0832-ec2f-4cb0-881a-1cb110559af1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:21.053439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca0832-ec2f-4cb0-881a-1cb110559af1 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=53b19d145ddf4f3c950098b6269b33e0, user.id=f2181a60abf646d28f09241ba111a2d3) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:21.053995 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca0832-ec2f-4cb0-881a-1cb110559af1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:21.055687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca0832-ec2f-4cb0-881a-1cb110559af1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:21.062810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 77/155] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:20 2022] GET /identity/v3/role_assignments?user.id=f2181a60abf646d28f09241ba111a2d3&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 608 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:21.090637 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6293d49-f3fd-4f81-b0eb-c963b6773548 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:21.091332 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6293d49-f3fd-4f81-b0eb-c963b6773548 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:21.091859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6293d49-f3fd-4f81-b0eb-c963b6773548 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:21.093598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 79/156] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:21.100072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2383b-1989-4f17-ab2d-3b4bad8eabcf None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:21.100363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2383b-1989-4f17-ab2d-3b4bad8eabcf None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:21.100643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2383b-1989-4f17-ab2d-3b4bad8eabcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:21.121521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-5ec2383b-1989-4f17-ab2d-3b4bad8eabcf None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:21.122724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-5ec2383b-1989-4f17-ab2d-3b4bad8eabcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:21.204075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 78/157] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:21.323833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6293d49-f3fd-4f81-b0eb-c963b6773548 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:21.435398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6293d49-f3fd-4f81-b0eb-c963b6773548 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:21.468492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:21.475803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4aec5a-5fb3-479d-9b57-3f9e9f29fc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:21.476734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4aec5a-5fb3-479d-9b57-3f9e9f29fc1d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:21.477213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4aec5a-5fb3-479d-9b57-3f9e9f29fc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:21.477697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4aec5a-5fb3-479d-9b57-3f9e9f29fc1d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:21.479093 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4aec5a-5fb3-479d-9b57-3f9e9f29fc1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:21.480824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4aec5a-5fb3-479d-9b57-3f9e9f29fc1d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:21.482788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 80/158] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:21.488568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2383b-1989-4f17-ab2d-3b4bad8eabcf None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:21.542585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:21.569044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4aec5a-5fb3-479d-9b57-3f9e9f29fc1d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:21.612917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2383b-1989-4f17-ab2d-3b4bad8eabcf None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:21.658289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b0640-63f8-45bd-82ba-8d9f6e30c2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:21.659090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b0640-63f8-45bd-82ba-8d9f6e30c2f2 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:21.659288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b0640-63f8-45bd-82ba-8d9f6e30c2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:21.659576 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b0640-63f8-45bd-82ba-8d9f6e30c2f2 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:21.660312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b0640-63f8-45bd-82ba-8d9f6e30c2f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:21.661762 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b0640-63f8-45bd-82ba-8d9f6e30c2f2 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:21.713719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4aec5a-5fb3-479d-9b57-3f9e9f29fc1d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:21.757509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e07f49-357a-4cd1-a627-ecbac44bfada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:21.758192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07f49-357a-4cd1-a627-ecbac44bfada None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:21.758476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07f49-357a-4cd1-a627-ecbac44bfada None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:21.758753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e07f49-357a-4cd1-a627-ecbac44bfada None admin] PATH_INFO: `/v3/roles/member` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:21.773390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e07f49-357a-4cd1-a627-ecbac44bfada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:21.775734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 79/159] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:03:21 2022] POST /identity/v3/users => generated 283 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-93e07f49-357a-4cd1-a627-ecbac44bfada None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:21.783562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:21.784817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:21.785679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 81/160] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:21.787671 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e07f49-357a-4cd1-a627-ecbac44bfada None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:21.858843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:21.866629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b64a19-5fe0-4fb8-818d-da3eff791211 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:21.866913 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b64a19-5fe0-4fb8-818d-da3eff791211 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:21.867102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b64a19-5fe0-4fb8-818d-da3eff791211 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:21.868652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 80/161] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:21.874543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff5172-1508-426e-aead-613c42e3bed6 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:21.875261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff5172-1508-426e-aead-613c42e3bed6 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:21.875725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff5172-1508-426e-aead-613c42e3bed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:21.897034 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-a7ff5172-1508-426e-aead-613c42e3bed6 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:21.898269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a7ff5172-1508-426e-aead-613c42e3bed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:21.912561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e07f49-357a-4cd1-a627-ecbac44bfada None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:21.951408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d1f99-a955-4d3a-97d2-9994124c9307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:21.952093 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d1f99-a955-4d3a-97d2-9994124c9307 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:21.952358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d1f99-a955-4d3a-97d2-9994124c9307 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:21.958068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d1f99-a955-4d3a-97d2-9994124c9307 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:21.958068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d1f99-a955-4d3a-97d2-9994124c9307 None admin] RBAC: Adding query filter params (name=member) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:21.958068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d1f99-a955-4d3a-97d2-9994124c9307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:21.958068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d1f99-a955-4d3a-97d2-9994124c9307 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:21.964989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 82/162] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:21.966217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e472a6-88a0-48f3-8134-cac4e3d97da3 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:21.966506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e472a6-88a0-48f3-8134-cac4e3d97da3 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:21.966775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e472a6-88a0-48f3-8134-cac4e3d97da3 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:21.971807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 83/163] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:21.971807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e472a6-88a0-48f3-8134-cac4e3d97da3 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:21.990064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 81/164] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:21.990571 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf7589-8e44-4ae0-8a35-72209d6ff1d8 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:21.992600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf7589-8e44-4ae0-8a35-72209d6ff1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:21.992600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf7589-8e44-4ae0-8a35-72209d6ff1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:22.014787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-bbbf7589-8e44-4ae0-8a35-72209d6ff1d8 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:22.019071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-bbbf7589-8e44-4ae0-8a35-72209d6ff1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:22.059712 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:22.111594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 82/165] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:22.120112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e472a6-88a0-48f3-8134-cac4e3d97da3 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:22.149238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:22.164625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c581c15-12df-4fcf-b47b-5a25158a3674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:22.165270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581c15-12df-4fcf-b47b-5a25158a3674 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:22.167098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581c15-12df-4fcf-b47b-5a25158a3674 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:22.167098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c581c15-12df-4fcf-b47b-5a25158a3674 None admin] PATH_INFO: `/v3/users/769b0a726d17470cb59ff5847d0b33a4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:22.167098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c581c15-12df-4fcf-b47b-5a25158a3674 None admin] RBAC: Authorizing `identity:get_user(user_id=769b0a726d17470cb59ff5847d0b33a4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:22.168588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c581c15-12df-4fcf-b47b-5a25158a3674 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:22.170180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 84/166] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:21 2022] GET /identity/v3/users/769b0a726d17470cb59ff5847d0b33a4 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:22.175501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf7589-8e44-4ae0-8a35-72209d6ff1d8 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:22.281226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c581c15-12df-4fcf-b47b-5a25158a3674 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:22.300513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf7589-8e44-4ae0-8a35-72209d6ff1d8 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:22.345663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449e5a9-1e5e-47ff-b4bf-8869fcb990c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:22.346882 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449e5a9-1e5e-47ff-b4bf-8869fcb990c3 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:22.348406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449e5a9-1e5e-47ff-b4bf-8869fcb990c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:22.348715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449e5a9-1e5e-47ff-b4bf-8869fcb990c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:22.387181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:22.409751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c581c15-12df-4fcf-b47b-5a25158a3674 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:22.431052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449e5a9-1e5e-47ff-b4bf-8869fcb990c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:22.433126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449e5a9-1e5e-47ff-b4bf-8869fcb990c3 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:22.456498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d4794-22e2-4fc9-8659-9cbc0459135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:22.457180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d4794-22e2-4fc9-8659-9cbc0459135e None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:22.457483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d4794-22e2-4fc9-8659-9cbc0459135e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:22.457740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d4794-22e2-4fc9-8659-9cbc0459135e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:22.469682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d4794-22e2-4fc9-8659-9cbc0459135e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-ce8d4794-22e2-4fc9-8659-9cbc0459135e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:22.475605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:22.476846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 85/167] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:03:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:03:22.477786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d4794-22e2-4fc9-8659-9cbc0459135e None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:22.548251 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:22.562707 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 83/168] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:22.563602 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449e5a9-1e5e-47ff-b4bf-8869fcb990c3 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:22.608941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d4794-22e2-4fc9-8659-9cbc0459135e None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:22.642563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:22.654433 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8904a7-c254-4413-8f1b-2028c4976579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:22.655295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8904a7-c254-4413-8f1b-2028c4976579 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:22.655713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8904a7-c254-4413-8f1b-2028c4976579 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:22.656146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8904a7-c254-4413-8f1b-2028c4976579 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:22.670341 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8904a7-c254-4413-8f1b-2028c4976579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-ea8904a7-c254-4413-8f1b-2028c4976579 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:22.678507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:22.679587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:22.680662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:22.683399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 86/169] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:22.684834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8904a7-c254-4413-8f1b-2028c4976579 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:22.695227 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449e5a9-1e5e-47ff-b4bf-8869fcb990c3 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:22.738832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3446786-222f-4831-b05d-06ce4deba1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:22.739684 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3446786-222f-4831-b05d-06ce4deba1cb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:22.740117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3446786-222f-4831-b05d-06ce4deba1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:22.740962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3446786-222f-4831-b05d-06ce4deba1cb None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:22.741807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3446786-222f-4831-b05d-06ce4deba1cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:22.742655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3446786-222f-4831-b05d-06ce4deba1cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:22.744483 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3446786-222f-4831-b05d-06ce4deba1cb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:22.756213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 84/170] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:03:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:22.760964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3446786-222f-4831-b05d-06ce4deba1cb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:22.819840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8904a7-c254-4413-8f1b-2028c4976579 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:22.837764 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:22.888752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3446786-222f-4831-b05d-06ce4deba1cb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:22.893206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8cf6f-f7fd-4d37-8167-a0ac667357eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:22.893910 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8cf6f-f7fd-4d37-8167-a0ac667357eb None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:22.894310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8cf6f-f7fd-4d37-8167-a0ac667357eb None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:22.895273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8cf6f-f7fd-4d37-8167-a0ac667357eb None admin] PATH_INFO: `/v3/projects/38017532123a473d9cc50c0c5839bd21` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:22.900541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8cf6f-f7fd-4d37-8167-a0ac667357eb None admin] RBAC: Authorizing `identity:get_project(project_id=38017532123a473d9cc50c0c5839bd21)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:22.902122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8cf6f-f7fd-4d37-8167-a0ac667357eb None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:22.907788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 87/171] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:22 2022] GET /identity/v3/projects/38017532123a473d9cc50c0c5839bd21 => generated 312 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:22.907788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8cf6f-f7fd-4d37-8167-a0ac667357eb None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:22.922482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:22.935452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3a948-d7a4-47a7-aa54-95796e74d1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:22.936309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3a948-d7a4-47a7-aa54-95796e74d1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:22.936752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3a948-d7a4-47a7-aa54-95796e74d1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:22.937690 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3a948-d7a4-47a7-aa54-95796e74d1f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:22.938641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3a948-d7a4-47a7-aa54-95796e74d1f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:22.939208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3a948-d7a4-47a7-aa54-95796e74d1f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:22.940752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3a948-d7a4-47a7-aa54-95796e74d1f5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:22.952462 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 85/172] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:22.953711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3a948-d7a4-47a7-aa54-95796e74d1f5 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:23.118688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8cf6f-f7fd-4d37-8167-a0ac667357eb None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:23.145992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3a948-d7a4-47a7-aa54-95796e74d1f5 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:23.165308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83e3e2-3ed0-41ea-9c0f-52aa6b9d5f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:23.166243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83e3e2-3ed0-41ea-9c0f-52aa6b9d5f5c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:23.167708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83e3e2-3ed0-41ea-9c0f-52aa6b9d5f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:23.168226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83e3e2-3ed0-41ea-9c0f-52aa6b9d5f5c None admin] PATH_INFO: `/v3/users/769b0a726d17470cb59ff5847d0b33a4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:23.170493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83e3e2-3ed0-41ea-9c0f-52aa6b9d5f5c None admin] RBAC: Authorizing `identity:get_user(user_id=769b0a726d17470cb59ff5847d0b33a4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:23.173603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83e3e2-3ed0-41ea-9c0f-52aa6b9d5f5c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:23.182226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 88/173] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:22 2022] GET /identity/v3/users/769b0a726d17470cb59ff5847d0b33a4 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:23.184201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83e3e2-3ed0-41ea-9c0f-52aa6b9d5f5c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:23.217491 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97831ef-e2f4-4449-b615-05d729c442a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:23.218448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97831ef-e2f4-4449-b615-05d729c442a6 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:23.219654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97831ef-e2f4-4449-b615-05d729c442a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:23.221333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97831ef-e2f4-4449-b615-05d729c442a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:23.222216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97831ef-e2f4-4449-b615-05d729c442a6 None admin] RBAC: Adding query filter params (role.id=f463de99ba174eab9396eef514e34e15, scope.project.id=38017532123a473d9cc50c0c5839bd21, user.id=769b0a726d17470cb59ff5847d0b33a4) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:23.222875 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97831ef-e2f4-4449-b615-05d729c442a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:23.225284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97831ef-e2f4-4449-b615-05d729c442a6 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:23.237457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 86/174] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:22 2022] GET /identity/v3/role_assignments?user.id=769b0a726d17470cb59ff5847d0b33a4&scope.project.id=38017532123a473d9cc50c0c5839bd21&role.id=f463de99ba174eab9396eef514e34e15 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:23.239129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97831ef-e2f4-4449-b615-05d729c442a6 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:23.279825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:23.355908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:23.405237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83e3e2-3ed0-41ea-9c0f-52aa6b9d5f5c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:23.472265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf47d4b-fdfc-4199-af51-5e5a198d63c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:23.472990 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf47d4b-fdfc-4199-af51-5e5a198d63c1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:23.473268 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf47d4b-fdfc-4199-af51-5e5a198d63c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:23.473687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf47d4b-fdfc-4199-af51-5e5a198d63c1 None admin] PATH_INFO: `/v3/users/1cb85440fc1a4f8a9646580b3e466d6d` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:23.476347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97831ef-e2f4-4449-b615-05d729c442a6 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:23.481969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf47d4b-fdfc-4199-af51-5e5a198d63c1 None admin] RBAC: Authorizing `identity:get_user(user_id=1cb85440fc1a4f8a9646580b3e466d6d)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:23.481969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf47d4b-fdfc-4199-af51-5e5a198d63c1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:23.481969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 89/175] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:23 2022] GET /identity/v3/users/1cb85440fc1a4f8a9646580b3e466d6d => generated 289 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:23.488951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf47d4b-fdfc-4199-af51-5e5a198d63c1 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:23.538854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497130e5-ab28-4624-8a85-21ec3ae94bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:23.539959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497130e5-ab28-4624-8a85-21ec3ae94bbe None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:23.541110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497130e5-ab28-4624-8a85-21ec3ae94bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:23.541710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497130e5-ab28-4624-8a85-21ec3ae94bbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:23.676988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497130e5-ab28-4624-8a85-21ec3ae94bbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:23.680468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497130e5-ab28-4624-8a85-21ec3ae94bbe None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:23.698651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf47d4b-fdfc-4199-af51-5e5a198d63c1 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:23.721376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:23.765237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98e0e4-639e-45c8-862c-2d3169e7abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:23.765883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98e0e4-639e-45c8-862c-2d3169e7abe4 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:23.766161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98e0e4-639e-45c8-862c-2d3169e7abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:23.766435 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98e0e4-639e-45c8-862c-2d3169e7abe4 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:23.768967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98e0e4-639e-45c8-862c-2d3169e7abe4 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:23.771226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98e0e4-639e-45c8-862c-2d3169e7abe4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:23.773211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 90/176] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:23 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:23.778453 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98e0e4-639e-45c8-862c-2d3169e7abe4 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:23.781615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:23.885792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 87/177] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:23.891548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497130e5-ab28-4624-8a85-21ec3ae94bbe None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:23.967093 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98e0e4-639e-45c8-862c-2d3169e7abe4 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:24.006745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497130e5-ab28-4624-8a85-21ec3ae94bbe None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:24.045248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4e684-be40-4fbe-9be9-c583a8c33a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:24.045888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e684-be40-4fbe-9be9-c583a8c33a22 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:24.046168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e684-be40-4fbe-9be9-c583a8c33a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:24.047002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d4e684-be40-4fbe-9be9-c583a8c33a22 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:24.048130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34fd0f1-783f-4ef7-b248-02c7cca83679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:24.048794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fd0f1-783f-4ef7-b248-02c7cca83679 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:24.049069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fd0f1-783f-4ef7-b248-02c7cca83679 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:24.049343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34fd0f1-783f-4ef7-b248-02c7cca83679 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0/users/1cb85440fc1a4f8a9646580b3e466d6d/roles/4f0b83503c8847079d53d3dd089d98cb` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:24.051242 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4e684-be40-4fbe-9be9-c583a8c33a22 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:24.052730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d4e684-be40-4fbe-9be9-c583a8c33a22 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:24.054716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 88/178] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:23 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:24.054853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fd0f1-783f-4ef7-b248-02c7cca83679 None admin] RBAC: Authorizing `identity:create_grant(project_id=53b19d145ddf4f3c950098b6269b33e0, user_id=1cb85440fc1a4f8a9646580b3e466d6d, role_id=4f0b83503c8847079d53d3dd089d98cb)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:24.056325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34fd0f1-783f-4ef7-b248-02c7cca83679 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:24.061189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4e684-be40-4fbe-9be9-c583a8c33a22 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:24.089945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:24.149577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 91/179] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:03:23 2022] PUT /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0/users/1cb85440fc1a4f8a9646580b3e466d6d/roles/4f0b83503c8847079d53d3dd089d98cb => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:24.194964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d4e684-be40-4fbe-9be9-c583a8c33a22 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:24.243932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7871a94-d920-4130-a85c-dd71ca8d3615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:24.244475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7871a94-d920-4130-a85c-dd71ca8d3615 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:24.244749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7871a94-d920-4130-a85c-dd71ca8d3615 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:24.245020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7871a94-d920-4130-a85c-dd71ca8d3615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:24.245762 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7871a94-d920-4130-a85c-dd71ca8d3615 None admin] RBAC: Adding query filter params (role.id=f8c4be33fa2f4e69a3de71a3c4d7b179, scope.project.id=53b19d145ddf4f3c950098b6269b33e0, user.id=769b0a726d17470cb59ff5847d0b33a4) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:24.246135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7871a94-d920-4130-a85c-dd71ca8d3615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:24.247649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7871a94-d920-4130-a85c-dd71ca8d3615 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:24.257779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 89/180] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:24 2022] GET /identity/v3/role_assignments?user.id=769b0a726d17470cb59ff5847d0b33a4&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:24.624434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90852ab-a633-4a78-9465-c11e6324556e None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:24.624821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90852ab-a633-4a78-9465-c11e6324556e None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:24.625002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90852ab-a633-4a78-9465-c11e6324556e None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:24.626533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 92/181] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:24.633889 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9871f-24bc-4810-92ab-6acc17b16a10 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:24.634175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9871f-24bc-4810-92ab-6acc17b16a10 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:24.634445 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9871f-24bc-4810-92ab-6acc17b16a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:24.654266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-82a9871f-24bc-4810-92ab-6acc17b16a10 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:24.655461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-82a9871f-24bc-4810-92ab-6acc17b16a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:24.729510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 90/182] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:25.019573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90852ab-a633-4a78-9465-c11e6324556e None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:25.069763 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:25.097356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be59d6-18dd-444d-b8a1-6c126a7e5e08 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.098581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be59d6-18dd-444d-b8a1-6c126a7e5e08 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.098864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be59d6-18dd-444d-b8a1-6c126a7e5e08 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.100299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 91/183] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:03:25.106001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67493-d261-4cf1-85e5-e41fbece2f5f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.106605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67493-d261-4cf1-85e5-e41fbece2f5f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.107098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e67493-d261-4cf1-85e5-e41fbece2f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.132740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-26e67493-d261-4cf1-85e5-e41fbece2f5f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:25.134011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90852ab-a633-4a78-9465-c11e6324556e None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:25.134423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-26e67493-d261-4cf1-85e5-e41fbece2f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:25.143099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:25.172650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27e27a-3edc-4d21-9282-8fcdcd221946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:25.173285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27e27a-3edc-4d21-9282-8fcdcd221946 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.173568 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27e27a-3edc-4d21-9282-8fcdcd221946 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.173891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27e27a-3edc-4d21-9282-8fcdcd221946 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.185183 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27e27a-3edc-4d21-9282-8fcdcd221946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-ab27e27a-3edc-4d21-9282-8fcdcd221946 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:25.191308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:25.192257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:25.193092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 93/184] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:03:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:25.194666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc672d66-b614-4206-8084-6209bf79a500 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.194955 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc672d66-b614-4206-8084-6209bf79a500 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.195232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc672d66-b614-4206-8084-6209bf79a500 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.200739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 94/185] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:25.200739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc672d66-b614-4206-8084-6209bf79a500 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:25.211225 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 92/186] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:25.212396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009940f-d9b2-4077-bf37-76170869525a None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.212683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009940f-d9b2-4077-bf37-76170869525a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.212952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009940f-d9b2-4077-bf37-76170869525a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.231486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-3009940f-d9b2-4077-bf37-76170869525a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:25.232684 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3009940f-d9b2-4077-bf37-76170869525a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:25.308987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 93/187] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:25.335259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc672d66-b614-4206-8084-6209bf79a500 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:25.337147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:25.371016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81d568-51c6-437c-94e9-1145c1bba8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:25.371952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81d568-51c6-437c-94e9-1145c1bba8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.372451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81d568-51c6-437c-94e9-1145c1bba8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.372936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81d568-51c6-437c-94e9-1145c1bba8f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.373820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81d568-51c6-437c-94e9-1145c1bba8f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:25.374317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81d568-51c6-437c-94e9-1145c1bba8f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:25.375928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81d568-51c6-437c-94e9-1145c1bba8f5 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:25.384254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 95/188] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:25.391823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009940f-d9b2-4077-bf37-76170869525a None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:25.467963 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:25.480779 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81d568-51c6-437c-94e9-1145c1bba8f5 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:25.505561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009940f-d9b2-4077-bf37-76170869525a None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:25.549254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce63b5-673a-4fa2-90e8-d4af91aa3b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:25.549936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce63b5-673a-4fa2-90e8-d4af91aa3b7b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.550285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce63b5-673a-4fa2-90e8-d4af91aa3b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.550658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce63b5-673a-4fa2-90e8-d4af91aa3b7b None admin] PATH_INFO: `/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.565385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce63b5-673a-4fa2-90e8-d4af91aa3b7b None admin] RBAC: Authorizing `identity:get_user(user_id=ed7ee75458a0459f94a96cc1eed7fb8e)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:25.566661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce63b5-673a-4fa2-90e8-d4af91aa3b7b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:25.568469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 94/189] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:25 2022] GET /identity/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:25.573851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce63b5-673a-4fa2-90e8-d4af91aa3b7b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:25.598763 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81d568-51c6-437c-94e9-1145c1bba8f5 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:25.635730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29312fd1-67c2-4ff9-ad05-92ad6228cecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:25.636391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29312fd1-67c2-4ff9-ad05-92ad6228cecf None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.636675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29312fd1-67c2-4ff9-ad05-92ad6228cecf None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.636946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29312fd1-67c2-4ff9-ad05-92ad6228cecf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.638342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29312fd1-67c2-4ff9-ad05-92ad6228cecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:25.639806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29312fd1-67c2-4ff9-ad05-92ad6228cecf None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:25.641359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 96/190] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:25.646554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29312fd1-67c2-4ff9-ad05-92ad6228cecf None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:25.679888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:25.685242 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce63b5-673a-4fa2-90e8-d4af91aa3b7b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:25.715647 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:25.721888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a29d6-21ab-4a30-8276-eb06ea0450b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:25.723110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a29d6-21ab-4a30-8276-eb06ea0450b2 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.723395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a29d6-21ab-4a30-8276-eb06ea0450b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.723673 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a29d6-21ab-4a30-8276-eb06ea0450b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.750467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29312fd1-67c2-4ff9-ad05-92ad6228cecf None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:25.786443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed54b-6629-4e51-888a-435dc58f2eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:25.787353 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed54b-6629-4e51-888a-435dc58f2eeb None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.787628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed54b-6629-4e51-888a-435dc58f2eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.787932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaed54b-6629-4e51-888a-435dc58f2eeb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.796933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a29d6-21ab-4a30-8276-eb06ea0450b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:25.798439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a29d6-21ab-4a30-8276-eb06ea0450b2 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:25.798557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaed54b-6629-4e51-888a-435dc58f2eeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-dcaed54b-6629-4e51-888a-435dc58f2eeb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:25.803359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:25.804255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:25.805043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 97/191] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:03:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:25.805701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed54b-6629-4e51-888a-435dc58f2eeb None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:25.901756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 95/192] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 16:03:25.902313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:25.904704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a29d6-21ab-4a30-8276-eb06ea0450b2 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:25.914367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaed54b-6629-4e51-888a-435dc58f2eeb None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:25.940700 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:25.950261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af3f37-230d-40db-afb2-42c4ae73cc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:25.953038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af3f37-230d-40db-afb2-42c4ae73cc26 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.953038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af3f37-230d-40db-afb2-42c4ae73cc26 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.953038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af3f37-230d-40db-afb2-42c4ae73cc26 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.953038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af3f37-230d-40db-afb2-42c4ae73cc26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:25.953622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af3f37-230d-40db-afb2-42c4ae73cc26 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:26.015718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a29d6-21ab-4a30-8276-eb06ea0450b2 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:26.048277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 98/193] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:03:25 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:03:26.050388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af3f37-230d-40db-afb2-42c4ae73cc26 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:26.051559 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff6392-f9aa-4943-9c08-8a273688133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:26.052214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff6392-f9aa-4943-9c08-8a273688133a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:26.052492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff6392-f9aa-4943-9c08-8a273688133a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:26.052769 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efff6392-f9aa-4943-9c08-8a273688133a None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:26.053453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff6392-f9aa-4943-9c08-8a273688133a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:26.053758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff6392-f9aa-4943-9c08-8a273688133a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:26.055206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efff6392-f9aa-4943-9c08-8a273688133a None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:26.065346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 96/194] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:03:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:26.071959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff6392-f9aa-4943-9c08-8a273688133a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:26.196403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af3f37-230d-40db-afb2-42c4ae73cc26 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:26.199974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:26.214093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efff6392-f9aa-4943-9c08-8a273688133a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:26.238987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa740878-50e8-45a5-93bd-13ebb388b657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:26.240713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa740878-50e8-45a5-93bd-13ebb388b657 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:26.240713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa740878-50e8-45a5-93bd-13ebb388b657 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:26.240713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa740878-50e8-45a5-93bd-13ebb388b657 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:26.242386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa740878-50e8-45a5-93bd-13ebb388b657 None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:26.243672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa740878-50e8-45a5-93bd-13ebb388b657 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:26.245335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 99/195] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:26 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:26.251333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa740878-50e8-45a5-93bd-13ebb388b657 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:26.264495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf880db9-44a8-437a-9573-522be57bf947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:26.265227 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf880db9-44a8-437a-9573-522be57bf947 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:26.265575 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf880db9-44a8-437a-9573-522be57bf947 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:26.265917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf880db9-44a8-437a-9573-522be57bf947 None admin] PATH_INFO: `/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:26.267133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf880db9-44a8-437a-9573-522be57bf947 None admin] RBAC: Authorizing `identity:get_user(user_id=ed7ee75458a0459f94a96cc1eed7fb8e)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:26.268685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf880db9-44a8-437a-9573-522be57bf947 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:26.270548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 97/196] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:26 2022] GET /identity/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:26.274122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:26.276974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf880db9-44a8-437a-9573-522be57bf947 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:26.372713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa740878-50e8-45a5-93bd-13ebb388b657 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:26.398093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf880db9-44a8-437a-9573-522be57bf947 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:26.423107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fa278c-1f68-4dc3-b196-d71daeb1532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:26.424018 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa278c-1f68-4dc3-b196-d71daeb1532d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:26.424884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa278c-1f68-4dc3-b196-d71daeb1532d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:26.425335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fa278c-1f68-4dc3-b196-d71daeb1532d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:26.427046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa278c-1f68-4dc3-b196-d71daeb1532d None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4, user.id=ed7ee75458a0459f94a96cc1eed7fb8e) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:26.427680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa278c-1f68-4dc3-b196-d71daeb1532d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:26.429707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fa278c-1f68-4dc3-b196-d71daeb1532d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:26.437805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 100/197] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:26 2022] GET /identity/v3/role_assignments?user.id=ed7ee75458a0459f94a96cc1eed7fb8e&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:26.444351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787aa87-6817-42d3-b2b3-3fb12bb8ef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:26.444987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787aa87-6817-42d3-b2b3-3fb12bb8ef6a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:26.445264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787aa87-6817-42d3-b2b3-3fb12bb8ef6a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:26.445594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787aa87-6817-42d3-b2b3-3fb12bb8ef6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:26.481587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:26.536673 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787aa87-6817-42d3-b2b3-3fb12bb8ef6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:26.538178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787aa87-6817-42d3-b2b3-3fb12bb8ef6a None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:26.642629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca236c7-a814-497b-b47f-191086ae563a None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:26.643010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca236c7-a814-497b-b47f-191086ae563a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:26.643298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca236c7-a814-497b-b47f-191086ae563a None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:26.645165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 101/198] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:26.650679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 98/199] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:26.652049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6d749-828c-45ad-b52c-5f03b2797beb None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:26.652497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6d749-828c-45ad-b52c-5f03b2797beb None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:26.652918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6d749-828c-45ad-b52c-5f03b2797beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:26.656119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787aa87-6817-42d3-b2b3-3fb12bb8ef6a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:26.681914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-9cf6d749-828c-45ad-b52c-5f03b2797beb None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:26.683401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9cf6d749-828c-45ad-b52c-5f03b2797beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:26.769540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:26.805985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 102/200] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:26.832573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787aa87-6817-42d3-b2b3-3fb12bb8ef6a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:26.875291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c77d1b-52db-4d5f-8f3b-f9a52752aa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:26.875949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c77d1b-52db-4d5f-8f3b-f9a52752aa82 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:26.876226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c77d1b-52db-4d5f-8f3b-f9a52752aa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:26.876496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c77d1b-52db-4d5f-8f3b-f9a52752aa82 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:26.877634 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c77d1b-52db-4d5f-8f3b-f9a52752aa82 None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:26.879121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c77d1b-52db-4d5f-8f3b-f9a52752aa82 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:26.881061 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 99/201] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:26 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:26.887012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6d749-828c-45ad-b52c-5f03b2797beb None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:26.901369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:27.033503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6d749-828c-45ad-b52c-5f03b2797beb None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:27.079269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30215d30-8668-4476-8863-bc258acf419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:27.079904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30215d30-8668-4476-8863-bc258acf419f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:27.080178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30215d30-8668-4476-8863-bc258acf419f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:27.080455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30215d30-8668-4476-8863-bc258acf419f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:27.081194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30215d30-8668-4476-8863-bc258acf419f None admin] RBAC: Adding query filter params (role.id=f8c4be33fa2f4e69a3de71a3c4d7b179, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4, user.id=ed7ee75458a0459f94a96cc1eed7fb8e) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:27.081500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30215d30-8668-4476-8863-bc258acf419f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:27.083489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30215d30-8668-4476-8863-bc258acf419f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:27.089849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 103/202] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:26 2022] GET /identity/v3/role_assignments?user.id=ed7ee75458a0459f94a96cc1eed7fb8e&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:27.215119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c77d1b-52db-4d5f-8f3b-f9a52752aa82 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:27.362015 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c77d1b-52db-4d5f-8f3b-f9a52752aa82 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:27.404178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:27.411481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479f5b7-60d2-4b95-812f-83d56f16590f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:27.412162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479f5b7-60d2-4b95-812f-83d56f16590f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:27.412733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479f5b7-60d2-4b95-812f-83d56f16590f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:27.413012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479f5b7-60d2-4b95-812f-83d56f16590f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:27.426856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479f5b7-60d2-4b95-812f-83d56f16590f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-e479f5b7-60d2-4b95-812f-83d56f16590f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:27.436004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:27.436973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:27.437761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 100/203] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:27.440111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30215d30-8668-4476-8863-bc258acf419f None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:27.560567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:27.587879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30215d30-8668-4476-8863-bc258acf419f None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:27.630534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11272dc2-2d30-4ec6-8ae1-2526d4809e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:27.631065 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11272dc2-2d30-4ec6-8ae1-2526d4809e69 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:27.631346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11272dc2-2d30-4ec6-8ae1-2526d4809e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:27.631622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11272dc2-2d30-4ec6-8ae1-2526d4809e69 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:27.632316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11272dc2-2d30-4ec6-8ae1-2526d4809e69 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:27.632619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11272dc2-2d30-4ec6-8ae1-2526d4809e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:27.633997 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11272dc2-2d30-4ec6-8ae1-2526d4809e69 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:27.647875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 104/204] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:27.653158 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479f5b7-60d2-4b95-812f-83d56f16590f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:27.829484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479f5b7-60d2-4b95-812f-83d56f16590f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:27.871316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92093b-09eb-4134-8830-0824212e6abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:27.872360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92093b-09eb-4134-8830-0824212e6abf None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:27.872848 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92093b-09eb-4134-8830-0824212e6abf None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:27.873324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92093b-09eb-4134-8830-0824212e6abf None admin] PATH_INFO: `/v3/users/1cb85440fc1a4f8a9646580b3e466d6d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:27.878139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92093b-09eb-4134-8830-0824212e6abf None admin] RBAC: Authorizing `identity:get_user(user_id=1cb85440fc1a4f8a9646580b3e466d6d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:27.880457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92093b-09eb-4134-8830-0824212e6abf None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:27.882350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 101/205] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:27 2022] GET /identity/v3/users/1cb85440fc1a4f8a9646580b3e466d6d => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:27.887787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11272dc2-2d30-4ec6-8ae1-2526d4809e69 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:28.071542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11272dc2-2d30-4ec6-8ae1-2526d4809e69 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:28.114591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3977b-f957-499b-ac30-f6b3b54b5909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:28.115336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3977b-f957-499b-ac30-f6b3b54b5909 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:28.115528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3977b-f957-499b-ac30-f6b3b54b5909 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:28.115859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3977b-f957-499b-ac30-f6b3b54b5909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:28.122544 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:28.204970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3977b-f957-499b-ac30-f6b3b54b5909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:28.206352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa3977b-f957-499b-ac30-f6b3b54b5909 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:28.319391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 105/206] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:28.324976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92093b-09eb-4134-8830-0824212e6abf None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:28.389180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:28.446607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92093b-09eb-4134-8830-0824212e6abf None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:28.487482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426aaf5-8d67-43ed-a067-3575f002c5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:28.488622 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426aaf5-8d67-43ed-a067-3575f002c5cb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:28.488901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426aaf5-8d67-43ed-a067-3575f002c5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:28.489178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426aaf5-8d67-43ed-a067-3575f002c5cb None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:28.492455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426aaf5-8d67-43ed-a067-3575f002c5cb None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:28.493907 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426aaf5-8d67-43ed-a067-3575f002c5cb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:28.495544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 102/207] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:28 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:28.501480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3977b-f957-499b-ac30-f6b3b54b5909 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:28.560011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:28.643387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3977b-f957-499b-ac30-f6b3b54b5909 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:28.691802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876aef51-3c1d-4aa0-9304-34558d23d7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:28.692795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876aef51-3c1d-4aa0-9304-34558d23d7ab None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:28.693300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876aef51-3c1d-4aa0-9304-34558d23d7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:28.693767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876aef51-3c1d-4aa0-9304-34558d23d7ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:28.694700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876aef51-3c1d-4aa0-9304-34558d23d7ab None admin] RBAC: Adding query filter params (role.id=4f0b83503c8847079d53d3dd089d98cb, scope.project.id=53b19d145ddf4f3c950098b6269b33e0, user.id=1cb85440fc1a4f8a9646580b3e466d6d) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:28.695209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876aef51-3c1d-4aa0-9304-34558d23d7ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:28.696890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876aef51-3c1d-4aa0-9304-34558d23d7ab None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:28.703689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 106/208] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:28 2022] GET /identity/v3/role_assignments?user.id=1cb85440fc1a4f8a9646580b3e466d6d&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=4f0b83503c8847079d53d3dd089d98cb => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:28.958359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883810dc-0e5a-4014-9bf9-99cb1ac6668e None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:28.958772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883810dc-0e5a-4014-9bf9-99cb1ac6668e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:28.959000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883810dc-0e5a-4014-9bf9-99cb1ac6668e None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:28.960804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 103/209] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:28.967259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5a782-8679-4b27-9c49-9b17ae4e9ab7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:28.968002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5a782-8679-4b27-9c49-9b17ae4e9ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:28.968483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5a782-8679-4b27-9c49-9b17ae4e9ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:28.969210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470525a0-5c80-45d5-a9a5-1e9963b054b3 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:28.969493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470525a0-5c80-45d5-a9a5-1e9963b054b3 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:28.969772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470525a0-5c80-45d5-a9a5-1e9963b054b3 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:28.971193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 104/210] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:03:28.978075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca75cc-decf-4deb-a174-18f32b2e3d3c None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:28.978402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca75cc-decf-4deb-a174-18f32b2e3d3c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:28.978635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca75cc-decf-4deb-a174-18f32b2e3d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:29.015407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-5db5a782-8679-4b27-9c49-9b17ae4e9ab7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:29.015407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5db5a782-8679-4b27-9c49-9b17ae4e9ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:29.019241 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-66ca75cc-decf-4deb-a174-18f32b2e3d3c None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:29.020425 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-66ca75cc-decf-4deb-a174-18f32b2e3d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:29.098240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 107/211] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:29.099870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb8822-785f-4711-9f63-05cd580a29eb None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:29.100352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb8822-785f-4711-9f63-05cd580a29eb None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:29.100928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb8822-785f-4711-9f63-05cd580a29eb None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:29.102706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 108/212] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:29.103807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 105/213] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:29.107605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547a87-2258-4f50-90dd-cc95bef4fa4e None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:29.107978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547a87-2258-4f50-90dd-cc95bef4fa4e None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:29.108322 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62547a87-2258-4f50-90dd-cc95bef4fa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:29.131287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-62547a87-2258-4f50-90dd-cc95bef4fa4e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:29.132584 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-62547a87-2258-4f50-90dd-cc95bef4fa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:29.168155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:29.206240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 109/214] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:29.399501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca75cc-decf-4deb-a174-18f32b2e3d3c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:29.401151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62547a87-2258-4f50-90dd-cc95bef4fa4e None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:29.469930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:29.509477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62547a87-2258-4f50-90dd-cc95bef4fa4e None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:29.517509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca75cc-decf-4deb-a174-18f32b2e3d3c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:29.547692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a5083-f681-412e-928a-4926313889ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:29.548614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5083-f681-412e-928a-4926313889ef None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:29.549038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5083-f681-412e-928a-4926313889ef None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:29.549469 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a5083-f681-412e-928a-4926313889ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:29.555138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44edcc3-ad5e-443b-907e-74902d50df32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:29.555782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edcc3-ad5e-443b-907e-74902d50df32 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:29.556062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edcc3-ad5e-443b-907e-74902d50df32 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:29.556335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44edcc3-ad5e-443b-907e-74902d50df32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:29.561206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a5083-f681-412e-928a-4926313889ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-4d5a5083-f681-412e-928a-4926313889ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:29.566202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:29.567271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:29.568257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:29.569739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44edcc3-ad5e-443b-907e-74902d50df32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:29.572252 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 110/215] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:03:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:29.572767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a5083-f681-412e-928a-4926313889ef None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-b44edcc3-ad5e-443b-907e-74902d50df32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:29.574520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:29.575344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 106/216] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:29.580749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44edcc3-ad5e-443b-907e-74902d50df32 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:29.602899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:29.683971 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a5083-f681-412e-928a-4926313889ef None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:29.687272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44edcc3-ad5e-443b-907e-74902d50df32 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:29.702833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:29.719731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03802e-7c79-45fa-82df-a03967d5962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:29.720381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03802e-7c79-45fa-82df-a03967d5962d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:29.720663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03802e-7c79-45fa-82df-a03967d5962d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:29.720973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03802e-7c79-45fa-82df-a03967d5962d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:29.722080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03802e-7c79-45fa-82df-a03967d5962d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:29.723079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb40f5-bb24-4d17-93dd-c76dd3e544ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:29.723529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03802e-7c79-45fa-82df-a03967d5962d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:29.723705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb40f5-bb24-4d17-93dd-c76dd3e544ad None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:29.723994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb40f5-bb24-4d17-93dd-c76dd3e544ad None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:29.724272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb40f5-bb24-4d17-93dd-c76dd3e544ad None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:29.724946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb40f5-bb24-4d17-93dd-c76dd3e544ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:29.725121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 111/217] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:29.725869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03802e-7c79-45fa-82df-a03967d5962d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:29.727963 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb40f5-bb24-4d17-93dd-c76dd3e544ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:29.730846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abb40f5-bb24-4d17-93dd-c76dd3e544ad None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:29.742280 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 107/218] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:29.743232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb40f5-bb24-4d17-93dd-c76dd3e544ad None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:29.820793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:29.833805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03802e-7c79-45fa-82df-a03967d5962d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:29.855709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abb40f5-bb24-4d17-93dd-c76dd3e544ad None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:29.869118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a2378-ae09-4344-b475-85f629435761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:29.869735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a2378-ae09-4344-b475-85f629435761 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:29.870016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a2378-ae09-4344-b475-85f629435761 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:29.870294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a2378-ae09-4344-b475-85f629435761 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:29.870962 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a2378-ae09-4344-b475-85f629435761 None admin] RBAC: Adding query filter params (name=member) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:29.871264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a2378-ae09-4344-b475-85f629435761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:29.872684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a2378-ae09-4344-b475-85f629435761 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:29.886236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 112/219] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:29.887118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a2378-ae09-4344-b475-85f629435761 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:29.890437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7fdd3-4fc5-439b-97ba-abea84d48787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:29.891868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7fdd3-4fc5-439b-97ba-abea84d48787 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:29.892153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7fdd3-4fc5-439b-97ba-abea84d48787 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:29.892426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7fdd3-4fc5-439b-97ba-abea84d48787 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:29.893081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7fdd3-4fc5-439b-97ba-abea84d48787 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:29.895098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab7fdd3-4fc5-439b-97ba-abea84d48787 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:29.991886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 108/220] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:03:29 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:03:29.991886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7fdd3-4fc5-439b-97ba-abea84d48787 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:29.992201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a2378-ae09-4344-b475-85f629435761 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:30.027151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:30.030359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4dd02-b274-4547-8f2f-9e4cd4fc8c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:30.031012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4dd02-b274-4547-8f2f-9e4cd4fc8c32 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:30.031291 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4dd02-b274-4547-8f2f-9e4cd4fc8c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:30.031569 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4dd02-b274-4547-8f2f-9e4cd4fc8c32 None admin] PATH_INFO: `/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:30.032810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4dd02-b274-4547-8f2f-9e4cd4fc8c32 None admin] RBAC: Authorizing `identity:get_user(user_id=ed7ee75458a0459f94a96cc1eed7fb8e)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:30.034289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4dd02-b274-4547-8f2f-9e4cd4fc8c32 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:30.036767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 113/221] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:29 2022] GET /identity/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:30.037965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941afe1-83b1-4781-bd75-8e19901cdb80 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:30.038249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941afe1-83b1-4781-bd75-8e19901cdb80 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:30.038524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941afe1-83b1-4781-bd75-8e19901cdb80 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:30.044531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 114/222] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:30.045611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941afe1-83b1-4781-bd75-8e19901cdb80 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:30.071826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:30.110433 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7fdd3-4fc5-439b-97ba-abea84d48787 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:30.147367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fa7e7-2d68-41d1-8768-f35067916d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:30.148346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fa7e7-2d68-41d1-8768-f35067916d03 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:30.148626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fa7e7-2d68-41d1-8768-f35067916d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:30.148901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fa7e7-2d68-41d1-8768-f35067916d03 None admin] PATH_INFO: `/v3/users/943b81a63fd848f281b09302d96d04ed` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:30.167225 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fa7e7-2d68-41d1-8768-f35067916d03 None admin] RBAC: Authorizing `identity:get_user(user_id=943b81a63fd848f281b09302d96d04ed)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:30.169083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fa7e7-2d68-41d1-8768-f35067916d03 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:30.171327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 109/223] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:29 2022] GET /identity/v3/users/943b81a63fd848f281b09302d96d04ed => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:30.172211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d191e-3997-4322-be38-2cd6c2b46d1e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:30.172498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d191e-3997-4322-be38-2cd6c2b46d1e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:30.172768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d191e-3997-4322-be38-2cd6c2b46d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:30.185307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941afe1-83b1-4781-bd75-8e19901cdb80 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:30.192572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0d3d191e-3997-4322-be38-2cd6c2b46d1e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:30.193801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0d3d191e-3997-4322-be38-2cd6c2b46d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:30.225161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37af27-eaf3-4f1f-a01f-906572ae02bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:30.225952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37af27-eaf3-4f1f-a01f-906572ae02bd None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:30.227133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37af27-eaf3-4f1f-a01f-906572ae02bd None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:30.227133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa37af27-eaf3-4f1f-a01f-906572ae02bd None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:30.229441 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37af27-eaf3-4f1f-a01f-906572ae02bd None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:30.230395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa37af27-eaf3-4f1f-a01f-906572ae02bd None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:30.232539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 115/224] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:30 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:30.233564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37af27-eaf3-4f1f-a01f-906572ae02bd None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:30.278678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 110/225] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:30.279349 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d191e-3997-4322-be38-2cd6c2b46d1e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:30.306872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:30.344260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:30.366835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa37af27-eaf3-4f1f-a01f-906572ae02bd None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:30.407588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d191e-3997-4322-be38-2cd6c2b46d1e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:30.408886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c83bd-5bdc-4e98-baf2-b74866f461b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:30.409998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c83bd-5bdc-4e98-baf2-b74866f461b4 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:30.410499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c83bd-5bdc-4e98-baf2-b74866f461b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:30.410987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c83bd-5bdc-4e98-baf2-b74866f461b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:30.449187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75230e2-65ee-48e0-8a80-3336cc5b473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:30.449854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75230e2-65ee-48e0-8a80-3336cc5b473f None admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:30.450127 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75230e2-65ee-48e0-8a80-3336cc5b473f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:30.450394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75230e2-65ee-48e0-8a80-3336cc5b473f None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/users/ed7ee75458a0459f94a96cc1eed7fb8e/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:30.453734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75230e2-65ee-48e0-8a80-3336cc5b473f None admin] RBAC: Authorizing `identity:create_grant(project_id=225b5e655135449ab5cb4e25c5c9fca4, user_id=ed7ee75458a0459f94a96cc1eed7fb8e, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:30.455172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75230e2-65ee-48e0-8a80-3336cc5b473f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:30.504019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c83bd-5bdc-4e98-baf2-b74866f461b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:30.505604 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c83bd-5bdc-4e98-baf2-b74866f461b4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:30.542200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 111/226] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:03:30 2022] PUT /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/users/ed7ee75458a0459f94a96cc1eed7fb8e/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:30.568969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75230e2-65ee-48e0-8a80-3336cc5b473f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:30.617084 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:30.640124 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 116/227] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:30.649204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c83bd-5bdc-4e98-baf2-b74866f461b4 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:30.736715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:30.776546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75230e2-65ee-48e0-8a80-3336cc5b473f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:30.826368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df4056-a60e-4db9-a363-e3fb9a0acdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:30.826368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df4056-a60e-4db9-a363-e3fb9a0acdad None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:30.826368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df4056-a60e-4db9-a363-e3fb9a0acdad None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:30.826679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c83bd-5bdc-4e98-baf2-b74866f461b4 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:30.826765 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df4056-a60e-4db9-a363-e3fb9a0acdad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:30.839599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df4056-a60e-4db9-a363-e3fb9a0acdad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-32df4056-a60e-4db9-a363-e3fb9a0acdad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:30.845344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:30.846224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:30.847059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 112/228] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:03:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:30.854402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df4056-a60e-4db9-a363-e3fb9a0acdad None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:30.865404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e383c-c73a-40e8-a6ae-13ffd9e92ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:30.866045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e383c-c73a-40e8-a6ae-13ffd9e92ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:30.866323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e383c-c73a-40e8-a6ae-13ffd9e92ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:30.866597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e383c-c73a-40e8-a6ae-13ffd9e92ac0 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:30.868197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e383c-c73a-40e8-a6ae-13ffd9e92ac0 None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:30.870051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e383c-c73a-40e8-a6ae-13ffd9e92ac0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:30.873181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 117/229] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:30 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:30.879566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e383c-c73a-40e8-a6ae-13ffd9e92ac0 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:30.974555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df4056-a60e-4db9-a363-e3fb9a0acdad None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:30.990161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:31.018751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d207f16-0095-46d7-8037-117a8782e306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:31.019419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207f16-0095-46d7-8037-117a8782e306 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:31.019693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207f16-0095-46d7-8037-117a8782e306 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:31.019988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d207f16-0095-46d7-8037-117a8782e306 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:31.020793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d207f16-0095-46d7-8037-117a8782e306 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:31.021096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d207f16-0095-46d7-8037-117a8782e306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:31.022649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d207f16-0095-46d7-8037-117a8782e306 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:31.031862 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 113/230] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:03:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:31.040859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d207f16-0095-46d7-8037-117a8782e306 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:31.043691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:31.059234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e383c-c73a-40e8-a6ae-13ffd9e92ac0 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:31.103125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ff393-646f-471e-a02f-b458155e8b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:31.103656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ff393-646f-471e-a02f-b458155e8b2d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:31.104162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ff393-646f-471e-a02f-b458155e8b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:31.104442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ff393-646f-471e-a02f-b458155e8b2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:31.107167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ff393-646f-471e-a02f-b458155e8b2d None admin] RBAC: Adding query filter params (role.id=f463de99ba174eab9396eef514e34e15, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4, user.id=943b81a63fd848f281b09302d96d04ed) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:31.107482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ff393-646f-471e-a02f-b458155e8b2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:31.109770 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ff393-646f-471e-a02f-b458155e8b2d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:31.124339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 118/231] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:30 2022] GET /identity/v3/role_assignments?user.id=943b81a63fd848f281b09302d96d04ed&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=f463de99ba174eab9396eef514e34e15 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:31.216992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d207f16-0095-46d7-8037-117a8782e306 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:31.265824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3720f6-3409-4fc5-a803-0e8f5da5e8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:31.266488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3720f6-3409-4fc5-a803-0e8f5da5e8ae None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:31.266771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3720f6-3409-4fc5-a803-0e8f5da5e8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:31.267048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3720f6-3409-4fc5-a803-0e8f5da5e8ae None admin] PATH_INFO: `/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:31.270971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3720f6-3409-4fc5-a803-0e8f5da5e8ae None admin] RBAC: Authorizing `identity:get_user(user_id=ed7ee75458a0459f94a96cc1eed7fb8e)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:31.272517 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3720f6-3409-4fc5-a803-0e8f5da5e8ae None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:31.274244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 114/232] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:31 2022] GET /identity/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:31.279965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ff393-646f-471e-a02f-b458155e8b2d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:31.406612 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ff393-646f-471e-a02f-b458155e8b2d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:31.423906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:31.442487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28cd20c-8b61-4838-868c-7229505d68c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:31.443122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd20c-8b61-4838-868c-7229505d68c2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:31.443399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd20c-8b61-4838-868c-7229505d68c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:31.443669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28cd20c-8b61-4838-868c-7229505d68c2 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:31.444868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28cd20c-8b61-4838-868c-7229505d68c2 None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:31.446298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28cd20c-8b61-4838-868c-7229505d68c2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:31.447935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 119/233] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:31 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:31.453988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3720f6-3409-4fc5-a803-0e8f5da5e8ae None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:31.539321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:31.577221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3720f6-3409-4fc5-a803-0e8f5da5e8ae None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:31.623050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01262847-ee4e-4eb1-b207-3ec43f082b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:31.623722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01262847-ee4e-4eb1-b207-3ec43f082b32 None admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:31.624022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01262847-ee4e-4eb1-b207-3ec43f082b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:31.624293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01262847-ee4e-4eb1-b207-3ec43f082b32 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/users/ed7ee75458a0459f94a96cc1eed7fb8e/roles/f8c4be33fa2f4e69a3de71a3c4d7b179` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:31.626440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01262847-ee4e-4eb1-b207-3ec43f082b32 None admin] RBAC: Authorizing `identity:create_grant(project_id=225b5e655135449ab5cb4e25c5c9fca4, user_id=ed7ee75458a0459f94a96cc1eed7fb8e, role_id=f8c4be33fa2f4e69a3de71a3c4d7b179)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:31.627900 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01262847-ee4e-4eb1-b207-3ec43f082b32 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:31.714607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 115/234] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:03:31 2022] PUT /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/users/ed7ee75458a0459f94a96cc1eed7fb8e/roles/f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:03:32.638243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48f7b5-58c1-4418-8704-9c8a012c7ba6 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:32.638758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48f7b5-58c1-4418-8704-9c8a012c7ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:32.638758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48f7b5-58c1-4418-8704-9c8a012c7ba6 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:32.640495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 120/235] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:32.646082 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144413ee-da62-44c6-9c89-893002e12009 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:32.646890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144413ee-da62-44c6-9c89-893002e12009 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:32.647356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144413ee-da62-44c6-9c89-893002e12009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:32.656278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a4962-612d-4077-be64-666767cb27fe None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:32.656278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a4962-612d-4077-be64-666767cb27fe None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:32.656278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a4962-612d-4077-be64-666767cb27fe None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:32.656278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 121/236] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:32.662270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90df4d-4178-403d-8344-85e233bde2d0 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:32.662558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90df4d-4178-403d-8344-85e233bde2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:32.662841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90df4d-4178-403d-8344-85e233bde2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:32.673580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-144413ee-da62-44c6-9c89-893002e12009 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:32.675058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-144413ee-da62-44c6-9c89-893002e12009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:32.682919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1e90df4d-4178-403d-8344-85e233bde2d0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:32.684139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1e90df4d-4178-403d-8344-85e233bde2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:32.762279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 122/237] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:32.791789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 116/238] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:33.091483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90df4d-4178-403d-8344-85e233bde2d0 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:33.118369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144413ee-da62-44c6-9c89-893002e12009 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:33.217315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:33.271348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90df4d-4178-403d-8344-85e233bde2d0 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:33.290863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144413ee-da62-44c6-9c89-893002e12009 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:33.309622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:33.347639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442a774-56e9-40c8-af84-967def02a851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:33.348319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442a774-56e9-40c8-af84-967def02a851 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:33.348592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442a774-56e9-40c8-af84-967def02a851 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:33.348864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442a774-56e9-40c8-af84-967def02a851 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:33.350634 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442a774-56e9-40c8-af84-967def02a851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:33.352141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442a774-56e9-40c8-af84-967def02a851 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:33.353709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 123/239] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:33.363281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442a774-56e9-40c8-af84-967def02a851 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:33.370408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c82a7-2d9a-4057-b7f4-3265a311e438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:33.371052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c82a7-2d9a-4057-b7f4-3265a311e438 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:33.371332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c82a7-2d9a-4057-b7f4-3265a311e438 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:33.371601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c82a7-2d9a-4057-b7f4-3265a311e438 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:33.389967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c82a7-2d9a-4057-b7f4-3265a311e438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-c40c82a7-2d9a-4057-b7f4-3265a311e438 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:33.403787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:33.404676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:33.405455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:33.406050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 117/240] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:03:33.412598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c82a7-2d9a-4057-b7f4-3265a311e438 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:33.516775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442a774-56e9-40c8-af84-967def02a851 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:33.543854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c82a7-2d9a-4057-b7f4-3265a311e438 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:33.568756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623276f-7dbc-419e-8378-bfd8f21a6d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:33.569694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623276f-7dbc-419e-8378-bfd8f21a6d87 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:33.570736 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:33.571177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623276f-7dbc-419e-8378-bfd8f21a6d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:33.572573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623276f-7dbc-419e-8378-bfd8f21a6d87 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:33.573632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623276f-7dbc-419e-8378-bfd8f21a6d87 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:33.575905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623276f-7dbc-419e-8378-bfd8f21a6d87 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:33.585245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a694a-301c-4b21-87bc-5d3d91e286c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:33.586433 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a694a-301c-4b21-87bc-5d3d91e286c0 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:33.586433 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a694a-301c-4b21-87bc-5d3d91e286c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:33.586704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a694a-301c-4b21-87bc-5d3d91e286c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:33.587375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a694a-301c-4b21-87bc-5d3d91e286c0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:33.587680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a694a-301c-4b21-87bc-5d3d91e286c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:33.590145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a694a-301c-4b21-87bc-5d3d91e286c0 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:33.604221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 118/241] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:33.604221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76425-6aa2-4af4-b59e-d8a6002f63fb None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:33.604221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76425-6aa2-4af4-b59e-d8a6002f63fb None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:33.604221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76425-6aa2-4af4-b59e-d8a6002f63fb None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:33.604221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 119/242] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:33.608161 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b76425-6aa2-4af4-b59e-d8a6002f63fb None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:33.702965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 124/243] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:03:33 2022] POST /identity/v3/users => generated 293 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:03:33.705184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccfc23-0b8c-4f3d-8ab0-571e3351f9be None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:33.705688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccfc23-0b8c-4f3d-8ab0-571e3351f9be None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:33.706838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bccfc23-0b8c-4f3d-8ab0-571e3351f9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:33.728593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b76425-6aa2-4af4-b59e-d8a6002f63fb None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:33.731940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-7bccfc23-0b8c-4f3d-8ab0-571e3351f9be None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:33.734559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7bccfc23-0b8c-4f3d-8ab0-571e3351f9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:33.802917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3ece1-2fa5-4473-a25c-9a6ac10afde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:33.803961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3ece1-2fa5-4473-a25c-9a6ac10afde0 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:33.804524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3ece1-2fa5-4473-a25c-9a6ac10afde0 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:33.805004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3ece1-2fa5-4473-a25c-9a6ac10afde0 None admin] PATH_INFO: `/v3/users/82d7cb6b93ed4ee4b70e061f7bf56874` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:33.818086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:33.822857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3ece1-2fa5-4473-a25c-9a6ac10afde0 None admin] RBAC: Authorizing `identity:get_user(user_id=82d7cb6b93ed4ee4b70e061f7bf56874)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:33.824461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3ece1-2fa5-4473-a25c-9a6ac10afde0 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:33.826554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 120/244] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:33 2022] GET /identity/v3/users/82d7cb6b93ed4ee4b70e061f7bf56874 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:33.834131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3ece1-2fa5-4473-a25c-9a6ac10afde0 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:33.904859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 125/245] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:33.928223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:34.001623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050edcc0-8047-4896-9a85-fd2947890763 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:34.001958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050edcc0-8047-4896-9a85-fd2947890763 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:34.002240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050edcc0-8047-4896-9a85-fd2947890763 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:34.004930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 126/246] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:34.012192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af14c1-79d1-4475-8713-36891059bacd None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:34.012564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af14c1-79d1-4475-8713-36891059bacd None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:34.013021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af14c1-79d1-4475-8713-36891059bacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:34.026173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3ece1-2fa5-4473-a25c-9a6ac10afde0 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:34.034242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-85af14c1-79d1-4475-8713-36891059bacd None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:34.035652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-85af14c1-79d1-4475-8713-36891059bacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:34.064865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1ca77-e73d-4fd4-a833-2878980bc30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:34.065406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1ca77-e73d-4fd4-a833-2878980bc30c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:34.066258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1ca77-e73d-4fd4-a833-2878980bc30c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:34.066526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1ca77-e73d-4fd4-a833-2878980bc30c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:34.122835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 127/247] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:34.151650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1ca77-e73d-4fd4-a833-2878980bc30c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:34.153137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1ca77-e73d-4fd4-a833-2878980bc30c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:34.197735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:34.256020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 121/248] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:34.261705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af14c1-79d1-4475-8713-36891059bacd None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:34.297626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1ca77-e73d-4fd4-a833-2878980bc30c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:34.384628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:34.388673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af14c1-79d1-4475-8713-36891059bacd None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:34.435064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1ca77-e73d-4fd4-a833-2878980bc30c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:34.436219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acff3ab-d78b-43ba-b00a-83b4ea2dc6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:34.437028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acff3ab-d78b-43ba-b00a-83b4ea2dc6cc None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:34.437555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acff3ab-d78b-43ba-b00a-83b4ea2dc6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:34.437972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acff3ab-d78b-43ba-b00a-83b4ea2dc6cc None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:34.440081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acff3ab-d78b-43ba-b00a-83b4ea2dc6cc None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:34.441723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acff3ab-d78b-43ba-b00a-83b4ea2dc6cc None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:34.443534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 128/249] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:34 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:34.444591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acff3ab-d78b-43ba-b00a-83b4ea2dc6cc None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:34.472727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e139414-2373-4026-bad7-a635009149d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:34.473516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e139414-2373-4026-bad7-a635009149d6 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:34.474222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e139414-2373-4026-bad7-a635009149d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:34.474640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e139414-2373-4026-bad7-a635009149d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:34.484936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:34.489001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e139414-2373-4026-bad7-a635009149d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-7e139414-2373-4026-bad7-a635009149d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:34.493837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:34.497608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 122/250] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:03:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:34.498407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e139414-2373-4026-bad7-a635009149d6 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:34.578244 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acff3ab-d78b-43ba-b00a-83b4ea2dc6cc None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:34.615549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e139414-2373-4026-bad7-a635009149d6 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:34.621749 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9334ba3-0b4b-49d9-bf84-9e2e1708a571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:34.622389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334ba3-0b4b-49d9-bf84-9e2e1708a571 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:34.622665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334ba3-0b4b-49d9-bf84-9e2e1708a571 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:34.622935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9334ba3-0b4b-49d9-bf84-9e2e1708a571 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:34.636321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9334ba3-0b4b-49d9-bf84-9e2e1708a571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-b9334ba3-0b4b-49d9-bf84-9e2e1708a571 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:34.642050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:34.642947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:34.644306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 129/251] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:34.644995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3f8d6-c152-41a5-9f99-acb217fc912f None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:34.645806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3f8d6-c152-41a5-9f99-acb217fc912f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:34.645806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed3f8d6-c152-41a5-9f99-acb217fc912f None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:34.648441 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 130/252] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:34.649452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3f8d6-c152-41a5-9f99-acb217fc912f None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:34.664037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990e018-0645-4db7-886d-bf9ee8613776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:34.665130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990e018-0645-4db7-886d-bf9ee8613776 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:34.665609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990e018-0645-4db7-886d-bf9ee8613776 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:34.666134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990e018-0645-4db7-886d-bf9ee8613776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:34.667067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990e018-0645-4db7-886d-bf9ee8613776 None admin] RBAC: Adding query filter params (role.id=4f0b83503c8847079d53d3dd089d98cb, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4, user.id=82d7cb6b93ed4ee4b70e061f7bf56874) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:34.667650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990e018-0645-4db7-886d-bf9ee8613776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:34.669465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1990e018-0645-4db7-886d-bf9ee8613776 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:34.675990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 123/253] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:34 2022] GET /identity/v3/role_assignments?user.id=82d7cb6b93ed4ee4b70e061f7bf56874&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=4f0b83503c8847079d53d3dd089d98cb => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:34.677053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990e018-0645-4db7-886d-bf9ee8613776 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:34.700869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:34.788335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:34.832162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990e018-0645-4db7-886d-bf9ee8613776 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:34.842174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed3f8d6-c152-41a5-9f99-acb217fc912f None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:34.888148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e795c6c-998d-478a-a4d7-12dc9385a361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:34.889238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e795c6c-998d-478a-a4d7-12dc9385a361 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:34.889873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e795c6c-998d-478a-a4d7-12dc9385a361 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:34.890353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e795c6c-998d-478a-a4d7-12dc9385a361 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:34.891735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e795c6c-998d-478a-a4d7-12dc9385a361 None admin] RBAC: Adding query filter params (name=member) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:34.892543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e795c6c-998d-478a-a4d7-12dc9385a361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:34.894778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e795c6c-998d-478a-a4d7-12dc9385a361 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:34.896811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab857e-4d3f-44f1-a160-1a22392e38be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:34.898605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab857e-4d3f-44f1-a160-1a22392e38be None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:34.898881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab857e-4d3f-44f1-a160-1a22392e38be None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:34.899152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab857e-4d3f-44f1-a160-1a22392e38be None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:34.899840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab857e-4d3f-44f1-a160-1a22392e38be None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:34.900144 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab857e-4d3f-44f1-a160-1a22392e38be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:34.901504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab857e-4d3f-44f1-a160-1a22392e38be None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:34.906256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 124/254] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:34.907715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69eb6dd-3a5d-4842-94dc-0a456c440398 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:34.908246 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69eb6dd-3a5d-4842-94dc-0a456c440398 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:34.908777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69eb6dd-3a5d-4842-94dc-0a456c440398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:34.924118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 131/255] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:34.925500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab857e-4d3f-44f1-a160-1a22392e38be None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:34.941427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-e69eb6dd-3a5d-4842-94dc-0a456c440398 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:34.944520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e69eb6dd-3a5d-4842-94dc-0a456c440398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:35.043704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab857e-4d3f-44f1-a160-1a22392e38be None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:35.048097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:35.064054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 125/256] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:35.064898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69eb6dd-3a5d-4842-94dc-0a456c440398 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:35.109181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc78a3e-7998-46ec-99ec-1c314131b4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:35.109929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc78a3e-7998-46ec-99ec-1c314131b4ee None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:35.110220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc78a3e-7998-46ec-99ec-1c314131b4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:35.110494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc78a3e-7998-46ec-99ec-1c314131b4ee None admin] PATH_INFO: `/v3/users/943b81a63fd848f281b09302d96d04ed` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:35.114462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc78a3e-7998-46ec-99ec-1c314131b4ee None admin] RBAC: Authorizing `identity:get_user(user_id=943b81a63fd848f281b09302d96d04ed)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:35.116227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc78a3e-7998-46ec-99ec-1c314131b4ee None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:35.118128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 132/257] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:34 2022] GET /identity/v3/users/943b81a63fd848f281b09302d96d04ed => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:35.127104 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:35.129194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc78a3e-7998-46ec-99ec-1c314131b4ee None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:35.172655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69eb6dd-3a5d-4842-94dc-0a456c440398 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:35.208609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed203354-ab3a-48a8-be43-d59456d2349c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:35.209257 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed203354-ab3a-48a8-be43-d59456d2349c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:35.209778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed203354-ab3a-48a8-be43-d59456d2349c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:35.210149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed203354-ab3a-48a8-be43-d59456d2349c None admin] PATH_INFO: `/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:35.211404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed203354-ab3a-48a8-be43-d59456d2349c None admin] RBAC: Authorizing `identity:get_user(user_id=ed7ee75458a0459f94a96cc1eed7fb8e)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:35.212989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed203354-ab3a-48a8-be43-d59456d2349c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:35.214965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 126/258] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:35 2022] GET /identity/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:35.220533 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed203354-ab3a-48a8-be43-d59456d2349c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:35.332602 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed203354-ab3a-48a8-be43-d59456d2349c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:35.361526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:35.370220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11764dd3-39d7-4512-b00c-41d2cf449682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:35.371422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764dd3-39d7-4512-b00c-41d2cf449682 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:35.371835 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764dd3-39d7-4512-b00c-41d2cf449682 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:35.372214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11764dd3-39d7-4512-b00c-41d2cf449682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:35.385120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc78a3e-7998-46ec-99ec-1c314131b4ee None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:35.447624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11764dd3-39d7-4512-b00c-41d2cf449682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:35.449200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11764dd3-39d7-4512-b00c-41d2cf449682 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:35.465015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83793b-bb83-43ac-80de-a3a310d3fb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:35.465670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83793b-bb83-43ac-80de-a3a310d3fb7f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:35.465949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83793b-bb83-43ac-80de-a3a310d3fb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:35.466225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83793b-bb83-43ac-80de-a3a310d3fb7f None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:35.472467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83793b-bb83-43ac-80de-a3a310d3fb7f None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:35.473931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83793b-bb83-43ac-80de-a3a310d3fb7f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:35.475584 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 133/259] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:35 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:35.485400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83793b-bb83-43ac-80de-a3a310d3fb7f None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:35.532447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:35.559744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 127/260] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 16:03:35.567003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11764dd3-39d7-4512-b00c-41d2cf449682 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:35.621369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:35.639314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83793b-bb83-43ac-80de-a3a310d3fb7f None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:35.678685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41564a-40c0-4295-b4f8-39bb7dfc18a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:35.679366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41564a-40c0-4295-b4f8-39bb7dfc18a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:35.679642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41564a-40c0-4295-b4f8-39bb7dfc18a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:35.679945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41564a-40c0-4295-b4f8-39bb7dfc18a2 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/users/943b81a63fd848f281b09302d96d04ed/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:35.682643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41564a-40c0-4295-b4f8-39bb7dfc18a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=225b5e655135449ab5cb4e25c5c9fca4, user_id=943b81a63fd848f281b09302d96d04ed, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:35.684434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41564a-40c0-4295-b4f8-39bb7dfc18a2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:35.733558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11764dd3-39d7-4512-b00c-41d2cf449682 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:35.770329 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc724975-d6e8-44c4-9828-18f11fe2fedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:35.771408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc724975-d6e8-44c4-9828-18f11fe2fedb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:35.771589 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 134/261] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:03:35 2022] PUT /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/users/943b81a63fd848f281b09302d96d04ed/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:35.772181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc724975-d6e8-44c4-9828-18f11fe2fedb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:35.772474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41564a-40c0-4295-b4f8-39bb7dfc18a2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:35.780832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc724975-d6e8-44c4-9828-18f11fe2fedb None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:35.780832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc724975-d6e8-44c4-9828-18f11fe2fedb None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:35.780832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc724975-d6e8-44c4-9828-18f11fe2fedb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:35.780832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 128/262] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:35 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:35.784742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc724975-d6e8-44c4-9828-18f11fe2fedb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:35.890300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:35.904985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc724975-d6e8-44c4-9828-18f11fe2fedb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:35.959218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ae762-f650-4658-beec-80d3e7f0839c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:35.959885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae762-f650-4658-beec-80d3e7f0839c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:35.961563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae762-f650-4658-beec-80d3e7f0839c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:35.961834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae762-f650-4658-beec-80d3e7f0839c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:35.962639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ae762-f650-4658-beec-80d3e7f0839c None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4, user.id=ed7ee75458a0459f94a96cc1eed7fb8e) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:35.967507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ae762-f650-4658-beec-80d3e7f0839c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:35.969127 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ae762-f650-4658-beec-80d3e7f0839c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:35.983287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 129/263] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:35 2022] GET /identity/v3/role_assignments?user.id=ed7ee75458a0459f94a96cc1eed7fb8e&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:36.019791 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:36.041319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41564a-40c0-4295-b4f8-39bb7dfc18a2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:36.082504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded2302-2372-4194-8933-9297fd9c8bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:36.083172 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded2302-2372-4194-8933-9297fd9c8bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:36.083451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded2302-2372-4194-8933-9297fd9c8bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:36.083789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded2302-2372-4194-8933-9297fd9c8bf0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:36.095709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded2302-2372-4194-8933-9297fd9c8bf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-bded2302-2372-4194-8933-9297fd9c8bf0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:36.101636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:36.102613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:36.103391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 135/264] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:03:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:36.118313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ae762-f650-4658-beec-80d3e7f0839c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:36.226861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ae762-f650-4658-beec-80d3e7f0839c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:36.268038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f17ea-531c-4dde-87c7-48ebf12a8a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:36.269083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f17ea-531c-4dde-87c7-48ebf12a8a51 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:36.269362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f17ea-531c-4dde-87c7-48ebf12a8a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:36.269631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f17ea-531c-4dde-87c7-48ebf12a8a51 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:36.270291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f17ea-531c-4dde-87c7-48ebf12a8a51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:36.270589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f17ea-531c-4dde-87c7-48ebf12a8a51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:36.271971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f17ea-531c-4dde-87c7-48ebf12a8a51 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:36.283562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 130/265] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:03:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:36.287088 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded2302-2372-4194-8933-9297fd9c8bf0 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:36.391708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded2302-2372-4194-8933-9297fd9c8bf0 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:36.426478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d820b1a-4095-49ac-b7af-32eadf56c4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:36.427111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d820b1a-4095-49ac-b7af-32eadf56c4fc None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:36.427391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d820b1a-4095-49ac-b7af-32eadf56c4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:36.427665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d820b1a-4095-49ac-b7af-32eadf56c4fc None admin] PATH_INFO: `/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:36.428831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d820b1a-4095-49ac-b7af-32eadf56c4fc None admin] RBAC: Authorizing `identity:get_user(user_id=ed7ee75458a0459f94a96cc1eed7fb8e)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:36.430280 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d820b1a-4095-49ac-b7af-32eadf56c4fc None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:36.432149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 136/266] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:36 2022] GET /identity/v3/users/ed7ee75458a0459f94a96cc1eed7fb8e => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:36.437965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f17ea-531c-4dde-87c7-48ebf12a8a51 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:36.463739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:36.551555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f17ea-531c-4dde-87c7-48ebf12a8a51 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:36.592146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5365d-419d-4241-b422-ad20922eb8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:36.592764 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5365d-419d-4241-b422-ad20922eb8bb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:36.593046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5365d-419d-4241-b422-ad20922eb8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:36.593321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5365d-419d-4241-b422-ad20922eb8bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:36.691663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5365d-419d-4241-b422-ad20922eb8bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:36.693166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5365d-419d-4241-b422-ad20922eb8bb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:36.716449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:36.825199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 131/267] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:36.830719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d820b1a-4095-49ac-b7af-32eadf56c4fc None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:36.857259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:36.971757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d820b1a-4095-49ac-b7af-32eadf56c4fc None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:37.065986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f79571-f159-4eca-8e64-af1c20fa742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:37.067429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f79571-f159-4eca-8e64-af1c20fa742b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:37.067930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f79571-f159-4eca-8e64-af1c20fa742b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:37.068398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f79571-f159-4eca-8e64-af1c20fa742b None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:37.069728 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f79571-f159-4eca-8e64-af1c20fa742b None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:37.071394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f79571-f159-4eca-8e64-af1c20fa742b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:37.073239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 137/268] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:36 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:37.079277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5365d-419d-4241-b422-ad20922eb8bb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:37.239671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5365d-419d-4241-b422-ad20922eb8bb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:37.266537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:37.290691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c35fdd3-92de-455f-8ca6-cea6e04459e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:37.291319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35fdd3-92de-455f-8ca6-cea6e04459e2 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:37.291605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35fdd3-92de-455f-8ca6-cea6e04459e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:37.291908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c35fdd3-92de-455f-8ca6-cea6e04459e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:37.292709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35fdd3-92de-455f-8ca6-cea6e04459e2 None admin] RBAC: Adding query filter params (role.id=f8c4be33fa2f4e69a3de71a3c4d7b179, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4, user.id=ed7ee75458a0459f94a96cc1eed7fb8e) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:37.293019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35fdd3-92de-455f-8ca6-cea6e04459e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:37.294543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c35fdd3-92de-455f-8ca6-cea6e04459e2 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:37.305508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 132/269] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:37 2022] GET /identity/v3/role_assignments?user.id=ed7ee75458a0459f94a96cc1eed7fb8e&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:37.342184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b841ecb-c487-4b80-9857-2e1549fc4714 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:37.342482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b841ecb-c487-4b80-9857-2e1549fc4714 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:37.342659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b841ecb-c487-4b80-9857-2e1549fc4714 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:37.344454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 138/270] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:37.350952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde26af4-025a-4b49-ad09-ca458c22475e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:37.351242 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde26af4-025a-4b49-ad09-ca458c22475e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:37.351526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde26af4-025a-4b49-ad09-ca458c22475e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:37.377349 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-fde26af4-025a-4b49-ad09-ca458c22475e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:37.378610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-fde26af4-025a-4b49-ad09-ca458c22475e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:37.457571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8387b-c21e-4b7a-9c81-6f1f4e19c573 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:37.458230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8387b-c21e-4b7a-9c81-6f1f4e19c573 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:37.458708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8387b-c21e-4b7a-9c81-6f1f4e19c573 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:37.460423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 139/271] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:37.466619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e35bb-e33c-491d-aa49-3d84435c1ad0 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:37.467265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e35bb-e33c-491d-aa49-3d84435c1ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:37.467740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e35bb-e33c-491d-aa49-3d84435c1ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:37.483799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 133/272] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:37.493587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-bf9e35bb-e33c-491d-aa49-3d84435c1ad0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:37.494807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-bf9e35bb-e33c-491d-aa49-3d84435c1ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:37.586967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 140/273] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:37.588915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf117952-578f-467e-bf9c-4d7cedc3ffe1 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:37.589206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf117952-578f-467e-bf9c-4d7cedc3ffe1 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:37.589485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf117952-578f-467e-bf9c-4d7cedc3ffe1 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:37.590847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 134/274] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:37.596395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e105e7b-8869-4adc-bdde-6317cfc7783f None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:37.597223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e105e7b-8869-4adc-bdde-6317cfc7783f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:37.597503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e105e7b-8869-4adc-bdde-6317cfc7783f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:37.617383 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-3e105e7b-8869-4adc-bdde-6317cfc7783f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:37.618638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-3e105e7b-8869-4adc-bdde-6317cfc7783f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:37.627281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:37.692783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 141/275] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:37.826210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf117952-578f-467e-bf9c-4d7cedc3ffe1 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:37.901083 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e105e7b-8869-4adc-bdde-6317cfc7783f None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:37.940038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf117952-578f-467e-bf9c-4d7cedc3ffe1 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:37.957757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:37.984908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1edc6-ca98-4657-a507-e300338aaed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:37.985811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1edc6-ca98-4657-a507-e300338aaed5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:37.986242 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1edc6-ca98-4657-a507-e300338aaed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:37.986662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1edc6-ca98-4657-a507-e300338aaed5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:37.988101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1edc6-ca98-4657-a507-e300338aaed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:37.990547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1edc6-ca98-4657-a507-e300338aaed5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:37.992936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 135/276] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:37.999587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1edc6-ca98-4657-a507-e300338aaed5 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:38.034406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e105e7b-8869-4adc-bdde-6317cfc7783f None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:38.078990 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21875f0-079d-4fcc-968e-323077be2ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:38.079927 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21875f0-079d-4fcc-968e-323077be2ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:38.080198 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21875f0-079d-4fcc-968e-323077be2ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:38.080476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21875f0-079d-4fcc-968e-323077be2ac7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:38.093364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21875f0-079d-4fcc-968e-323077be2ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-e21875f0-079d-4fcc-968e-323077be2ac7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:38.100232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:38.101150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 142/277] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:38.107203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21875f0-079d-4fcc-968e-323077be2ac7 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:38.109636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:38.117570 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1edc6-ca98-4657-a507-e300338aaed5 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:38.158458 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aab35-3517-45f0-b4ee-4a5e13c8f010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:38.159178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aab35-3517-45f0-b4ee-4a5e13c8f010 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:38.159460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aab35-3517-45f0-b4ee-4a5e13c8f010 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:38.160053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89aab35-3517-45f0-b4ee-4a5e13c8f010 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:38.160713 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89aab35-3517-45f0-b4ee-4a5e13c8f010 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:38.162156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89aab35-3517-45f0-b4ee-4a5e13c8f010 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:38.209482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21875f0-079d-4fcc-968e-323077be2ac7 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:38.239787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 136/278] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:03:37 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:03:38.239787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aab35-3517-45f0-b4ee-4a5e13c8f010 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:38.247798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748268b6-d0e5-4194-9be2-a8893affda51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:38.248552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748268b6-d0e5-4194-9be2-a8893affda51 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:38.248835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748268b6-d0e5-4194-9be2-a8893affda51 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:38.249113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748268b6-d0e5-4194-9be2-a8893affda51 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:38.265081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748268b6-d0e5-4194-9be2-a8893affda51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-748268b6-d0e5-4194-9be2-a8893affda51 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:38.270558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:38.271517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:38.272333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 143/279] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:38.277932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748268b6-d0e5-4194-9be2-a8893affda51 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:38.281220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:38.352811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:38.367367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89aab35-3517-45f0-b4ee-4a5e13c8f010 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:38.398623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748268b6-d0e5-4194-9be2-a8893affda51 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:38.409830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108ab0e-81ee-42b5-bca4-6e103de1a68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:38.410481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108ab0e-81ee-42b5-bca4-6e103de1a68e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:38.410764 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108ab0e-81ee-42b5-bca4-6e103de1a68e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:38.411039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108ab0e-81ee-42b5-bca4-6e103de1a68e None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:38.411711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108ab0e-81ee-42b5-bca4-6e103de1a68e None admin] RBAC: Adding query filter params (name=member) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:38.412156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108ab0e-81ee-42b5-bca4-6e103de1a68e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:38.413979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108ab0e-81ee-42b5-bca4-6e103de1a68e None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:38.424375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 137/280] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:38.430739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108ab0e-81ee-42b5-bca4-6e103de1a68e None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:38.437832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6571db24-3798-47c3-a6e7-7343f520e49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:38.438442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571db24-3798-47c3-a6e7-7343f520e49b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:38.438723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571db24-3798-47c3-a6e7-7343f520e49b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:38.438997 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6571db24-3798-47c3-a6e7-7343f520e49b None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:38.439693 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571db24-3798-47c3-a6e7-7343f520e49b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:38.440010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571db24-3798-47c3-a6e7-7343f520e49b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:38.441970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6571db24-3798-47c3-a6e7-7343f520e49b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:38.451957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 144/281] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:38.453209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99696fc5-a684-437d-b9c0-1d4705a473d4 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:38.453492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99696fc5-a684-437d-b9c0-1d4705a473d4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:38.453769 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99696fc5-a684-437d-b9c0-1d4705a473d4 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:38.454977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 145/282] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:38.460175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99696fc5-a684-437d-b9c0-1d4705a473d4 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:38.548140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:38.550981 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108ab0e-81ee-42b5-bca4-6e103de1a68e None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:38.588889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99696fc5-a684-437d-b9c0-1d4705a473d4 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:38.596316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896dda15-ee2d-4fe4-b048-72fa5dc3da26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:38.597774 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dda15-ee2d-4fe4-b048-72fa5dc3da26 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:38.597774 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dda15-ee2d-4fe4-b048-72fa5dc3da26 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:38.597774 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896dda15-ee2d-4fe4-b048-72fa5dc3da26 None admin] PATH_INFO: `/v3/users/2950b27761714033950e5954dfce0524` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:38.613237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896dda15-ee2d-4fe4-b048-72fa5dc3da26 None admin] RBAC: Authorizing `identity:get_user(user_id=2950b27761714033950e5954dfce0524)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:38.614281 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896dda15-ee2d-4fe4-b048-72fa5dc3da26 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:38.617786 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 138/283] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:38 2022] GET /identity/v3/users/2950b27761714033950e5954dfce0524 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:38.618434 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941e8b6-d187-4f34-81d4-16ed0f4fc323 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:38.618716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941e8b6-d187-4f34-81d4-16ed0f4fc323 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:38.618989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941e8b6-d187-4f34-81d4-16ed0f4fc323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:38.625563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:38.640455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7941e8b6-d187-4f34-81d4-16ed0f4fc323 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:38.641660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7941e8b6-d187-4f34-81d4-16ed0f4fc323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:38.644397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9e11d-bf43-4aa5-b74c-76577af74e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:38.646086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9e11d-bf43-4aa5-b74c-76577af74e41 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:38.646445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9e11d-bf43-4aa5-b74c-76577af74e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:38.646653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9e11d-bf43-4aa5-b74c-76577af74e41 None admin] PATH_INFO: `/v3/users/82d7cb6b93ed4ee4b70e061f7bf56874` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:38.647879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9e11d-bf43-4aa5-b74c-76577af74e41 None admin] RBAC: Authorizing `identity:get_user(user_id=82d7cb6b93ed4ee4b70e061f7bf56874)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:38.649341 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9e11d-bf43-4aa5-b74c-76577af74e41 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:38.652178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 146/284] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:38 2022] GET /identity/v3/users/82d7cb6b93ed4ee4b70e061f7bf56874 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:38.652997 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9e11d-bf43-4aa5-b74c-76577af74e41 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:38.718180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 139/285] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:38.719054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941e8b6-d187-4f34-81d4-16ed0f4fc323 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:38.804259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9e11d-bf43-4aa5-b74c-76577af74e41 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:38.863624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682c5aa-262d-483d-a476-4844560549d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:38.866414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682c5aa-262d-483d-a476-4844560549d3 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:38.866695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682c5aa-262d-483d-a476-4844560549d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:38.866965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682c5aa-262d-483d-a476-4844560549d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:38.867695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682c5aa-262d-483d-a476-4844560549d3 None admin] RBAC: Adding query filter params (role.id=f463de99ba174eab9396eef514e34e15, user.id=2950b27761714033950e5954dfce0524, scope.system=all) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:38.868025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682c5aa-262d-483d-a476-4844560549d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:38.871807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:38.872327 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c682c5aa-262d-483d-a476-4844560549d3 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:38.886478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 147/286] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Sep 7 16:03:38 2022] GET /identity/v3/role_assignments?user.id=2950b27761714033950e5954dfce0524&scope.system=all&role.id=f463de99ba174eab9396eef514e34e15 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:38.890676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941e8b6-d187-4f34-81d4-16ed0f4fc323 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:38.949831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d0510-81e8-4050-98fe-6b133f1d444f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:38.953296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d0510-81e8-4050-98fe-6b133f1d444f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:38.953296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d0510-81e8-4050-98fe-6b133f1d444f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:38.953296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d0510-81e8-4050-98fe-6b133f1d444f None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:38.953296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d0510-81e8-4050-98fe-6b133f1d444f None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:38.953698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d0510-81e8-4050-98fe-6b133f1d444f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:38.960292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 140/287] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:38 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:38.961326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682c5aa-262d-483d-a476-4844560549d3 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:39.026914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d0510-81e8-4050-98fe-6b133f1d444f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:39.067284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:39.136912 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682c5aa-262d-483d-a476-4844560549d3 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:39.170036 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d0510-81e8-4050-98fe-6b133f1d444f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:39.194112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a0ed7-e896-4a87-99fc-585305aa91fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:39.194793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a0ed7-e896-4a87-99fc-585305aa91fa None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:39.195074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a0ed7-e896-4a87-99fc-585305aa91fa None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:39.195352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445a0ed7-e896-4a87-99fc-585305aa91fa None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/users/82d7cb6b93ed4ee4b70e061f7bf56874/roles/4f0b83503c8847079d53d3dd089d98cb` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:39.198190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a0ed7-e896-4a87-99fc-585305aa91fa None admin] RBAC: Authorizing `identity:create_grant(project_id=225b5e655135449ab5cb4e25c5c9fca4, user_id=82d7cb6b93ed4ee4b70e061f7bf56874, role_id=4f0b83503c8847079d53d3dd089d98cb)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:39.199646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445a0ed7-e896-4a87-99fc-585305aa91fa None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:39.225209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c272c-d0ef-4a5f-a05b-e47620aea955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:39.225843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c272c-d0ef-4a5f-a05b-e47620aea955 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:39.226122 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c272c-d0ef-4a5f-a05b-e47620aea955 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:39.226407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c272c-d0ef-4a5f-a05b-e47620aea955 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:39.237375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c272c-d0ef-4a5f-a05b-e47620aea955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-5f1c272c-d0ef-4a5f-a05b-e47620aea955 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:39.242272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:39.243172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:39.244443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 141/288] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:39.251250 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c272c-d0ef-4a5f-a05b-e47620aea955 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:39.267921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:39.293404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 148/289] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:03:38 2022] PUT /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/users/82d7cb6b93ed4ee4b70e061f7bf56874/roles/4f0b83503c8847079d53d3dd089d98cb => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:03:39.375855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c272c-d0ef-4a5f-a05b-e47620aea955 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:39.433493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeaed1-6b2e-44e1-91ed-692004e6ac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:39.434122 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeaed1-6b2e-44e1-91ed-692004e6ac6c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:39.434397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeaed1-6b2e-44e1-91ed-692004e6ac6c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:39.434671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eeaed1-6b2e-44e1-91ed-692004e6ac6c None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:39.435345 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eeaed1-6b2e-44e1-91ed-692004e6ac6c None admin] RBAC: Adding query filter params (name=member) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:39.435646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eeaed1-6b2e-44e1-91ed-692004e6ac6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:39.437155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eeaed1-6b2e-44e1-91ed-692004e6ac6c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:39.451453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 142/290] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:39.457874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a0ed7-e896-4a87-99fc-585305aa91fa None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:39.590427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445a0ed7-e896-4a87-99fc-585305aa91fa None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:39.607505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:39.625455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7a5a8-6f7b-4305-a05d-8491a5668a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:39.626100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7a5a8-6f7b-4305-a05d-8491a5668a5c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:39.626384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7a5a8-6f7b-4305-a05d-8491a5668a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:39.626663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7a5a8-6f7b-4305-a05d-8491a5668a5c None admin] PATH_INFO: `/v3/users/943b81a63fd848f281b09302d96d04ed` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:39.628362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7a5a8-6f7b-4305-a05d-8491a5668a5c None admin] RBAC: Authorizing `identity:get_user(user_id=943b81a63fd848f281b09302d96d04ed)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:39.629850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7a5a8-6f7b-4305-a05d-8491a5668a5c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:39.631732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 149/291] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:39 2022] GET /identity/v3/users/943b81a63fd848f281b09302d96d04ed => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:39.637248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeaed1-6b2e-44e1-91ed-692004e6ac6c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:39.728050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:39.766619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eeaed1-6b2e-44e1-91ed-692004e6ac6c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:39.813697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5716f-431e-4fb3-9c96-cd802276f27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:39.814690 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5716f-431e-4fb3-9c96-cd802276f27a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:39.815226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5716f-431e-4fb3-9c96-cd802276f27a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:39.816259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d5716f-431e-4fb3-9c96-cd802276f27a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:39.911710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5716f-431e-4fb3-9c96-cd802276f27a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:39.913583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d5716f-431e-4fb3-9c96-cd802276f27a None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:40.021053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:40.059106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 143/292] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:40.064780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7a5a8-6f7b-4305-a05d-8491a5668a5c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:40.194142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7a5a8-6f7b-4305-a05d-8491a5668a5c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:40.240638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2fca3d-95e8-4cc8-8b1d-5bb9e10ea66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:40.241619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fca3d-95e8-4cc8-8b1d-5bb9e10ea66f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:40.242122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fca3d-95e8-4cc8-8b1d-5bb9e10ea66f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:40.242595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2fca3d-95e8-4cc8-8b1d-5bb9e10ea66f None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:40.244325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fca3d-95e8-4cc8-8b1d-5bb9e10ea66f None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:40.246138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2fca3d-95e8-4cc8-8b1d-5bb9e10ea66f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:40.248023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 150/293] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:40 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:40.254384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5716f-431e-4fb3-9c96-cd802276f27a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:40.410018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d5716f-431e-4fb3-9c96-cd802276f27a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:40.446151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab66276-2c77-4cd6-8c1b-1b4948b31bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:40.446646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab66276-2c77-4cd6-8c1b-1b4948b31bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:40.446923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab66276-2c77-4cd6-8c1b-1b4948b31bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:40.447191 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab66276-2c77-4cd6-8c1b-1b4948b31bb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:40.447946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab66276-2c77-4cd6-8c1b-1b4948b31bb8 None admin] RBAC: Adding query filter params (role.id=f463de99ba174eab9396eef514e34e15, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4, user.id=943b81a63fd848f281b09302d96d04ed) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:40.448251 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab66276-2c77-4cd6-8c1b-1b4948b31bb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:40.449696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab66276-2c77-4cd6-8c1b-1b4948b31bb8 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:40.456695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 144/294] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:40 2022] GET /identity/v3/role_assignments?user.id=943b81a63fd848f281b09302d96d04ed&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=f463de99ba174eab9396eef514e34e15 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:40.976789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6571f5-39c5-49a6-b352-2d2fb31178d4 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.011946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6571f5-39c5-49a6-b352-2d2fb31178d4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.011946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6571f5-39c5-49a6-b352-2d2fb31178d4 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.011946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 151/295] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:41.012720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ced3d-21ca-4418-aefd-c02e6e163ee8 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.012720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ced3d-21ca-4418-aefd-c02e6e163ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.012720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1ced3d-21ca-4418-aefd-c02e6e163ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.012938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7e1ced3d-21ca-4418-aefd-c02e6e163ee8 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:41.014035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7e1ced3d-21ca-4418-aefd-c02e6e163ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:41.051975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:41.089777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 145/296] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:41.292225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8eca13-9845-4122-907e-b871961ced40 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.292715 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8eca13-9845-4122-907e-b871961ced40 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.292715 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8eca13-9845-4122-907e-b871961ced40 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.294304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 152/297] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:41.300133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18261ce-ad83-4e9c-82b2-db0e521bb0d8 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.300542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18261ce-ad83-4e9c-82b2-db0e521bb0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.300714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18261ce-ad83-4e9c-82b2-db0e521bb0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.319619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-a18261ce-ad83-4e9c-82b2-db0e521bb0d8 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:41.320828 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a18261ce-ad83-4e9c-82b2-db0e521bb0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:41.384910 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8eca13-9845-4122-907e-b871961ced40 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:41.388945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 146/298] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:41.450477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:41.503266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8eca13-9845-4122-907e-b871961ced40 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:41.537996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac1052-22de-4b3b-9ac1-1b96f0240072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:41.539011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac1052-22de-4b3b-9ac1-1b96f0240072 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.539594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac1052-22de-4b3b-9ac1-1b96f0240072 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.540094 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac1052-22de-4b3b-9ac1-1b96f0240072 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.541444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac1052-22de-4b3b-9ac1-1b96f0240072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:41.543213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac1052-22de-4b3b-9ac1-1b96f0240072 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:41.545087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 153/299] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:41.551337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18261ce-ad83-4e9c-82b2-db0e521bb0d8 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:41.573206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cacc4-e3b1-4af0-aa1d-eb24124a613c None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.573537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cacc4-e3b1-4af0-aa1d-eb24124a613c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.573757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903cacc4-e3b1-4af0-aa1d-eb24124a613c None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.575313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 154/300] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:03:41.580201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd26e2-acb2-423b-8742-2506029f0d83 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.580487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd26e2-acb2-423b-8742-2506029f0d83 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.580793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd26e2-acb2-423b-8742-2506029f0d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.598848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1dcd26e2-acb2-423b-8742-2506029f0d83 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:41.600070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1dcd26e2-acb2-423b-8742-2506029f0d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:41.672334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:41.673217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 155/301] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:41.675392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd26e2-acb2-423b-8742-2506029f0d83 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:41.679525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18261ce-ad83-4e9c-82b2-db0e521bb0d8 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:41.720241 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ff036-a612-43b3-82b9-1b18788851ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:41.722450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ff036-a612-43b3-82b9-1b18788851ce None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.722760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ff036-a612-43b3-82b9-1b18788851ce None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.723037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ff036-a612-43b3-82b9-1b18788851ce None admin] PATH_INFO: `/v3/groups` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.723697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ff036-a612-43b3-82b9-1b18788851ce None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:41.725269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ff036-a612-43b3-82b9-1b18788851ce None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:41.743929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:41.792143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd26e2-acb2-423b-8742-2506029f0d83 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:41.807597 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 147/302] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Sep 7 16:03:41 2022] POST /identity/v3/groups => generated 229 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:03:41.814038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89cf57-90ad-4c9d-b85a-2db52fdcf0f6 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.814038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89cf57-90ad-4c9d-b85a-2db52fdcf0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.814330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89cf57-90ad-4c9d-b85a-2db52fdcf0f6 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.820226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 148/303] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:41.821574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225ec75-7fac-4fd7-83eb-150a937ef5b7 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.821856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225ec75-7fac-4fd7-83eb-150a937ef5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.822132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225ec75-7fac-4fd7-83eb-150a937ef5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.839368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9272b5e-bc14-4b7d-b0e4-d1a0c0caef6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:41.840481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9272b5e-bc14-4b7d-b0e4-d1a0c0caef6d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.840980 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9272b5e-bc14-4b7d-b0e4-d1a0c0caef6d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.841454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9272b5e-bc14-4b7d-b0e4-d1a0c0caef6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.841740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-e225ec75-7fac-4fd7-83eb-150a937ef5b7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:41.842955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e225ec75-7fac-4fd7-83eb-150a937ef5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:41.858968 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9272b5e-bc14-4b7d-b0e4-d1a0c0caef6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-a9272b5e-bc14-4b7d-b0e4-d1a0c0caef6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:41.866619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:41.867730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:41.873776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 156/304] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:41.875504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9272b5e-bc14-4b7d-b0e4-d1a0c0caef6d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:41.928612 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 149/305] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:41.945771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225ec75-7fac-4fd7-83eb-150a937ef5b7 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:42.025957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9272b5e-bc14-4b7d-b0e4-d1a0c0caef6d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:42.040677 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:42.059007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:42.069465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225ec75-7fac-4fd7-83eb-150a937ef5b7 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:42.074589 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e818f-902e-4980-838e-b338077ff73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:42.075457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e818f-902e-4980-838e-b338077ff73e None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:42.075989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e818f-902e-4980-838e-b338077ff73e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:42.076492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e818f-902e-4980-838e-b338077ff73e None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:42.077841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e818f-902e-4980-838e-b338077ff73e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:42.087298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e818f-902e-4980-838e-b338077ff73e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:42.087298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e818f-902e-4980-838e-b338077ff73e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:42.092521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 157/306] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:42.098943 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e818f-902e-4980-838e-b338077ff73e None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:42.120434 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be58d7-4ef7-4d9e-ba51-fff6fca727a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:42.121321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be58d7-4ef7-4d9e-ba51-fff6fca727a3 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:42.121606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be58d7-4ef7-4d9e-ba51-fff6fca727a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:42.121880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be58d7-4ef7-4d9e-ba51-fff6fca727a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:42.132776 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be58d7-4ef7-4d9e-ba51-fff6fca727a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-17be58d7-4ef7-4d9e-ba51-fff6fca727a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:42.138194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:42.139154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:42.144160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 150/307] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:03:42.146701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be58d7-4ef7-4d9e-ba51-fff6fca727a3 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:42.226202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e818f-902e-4980-838e-b338077ff73e None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:42.259788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d06fa1-6e9e-45f4-b5e1-2b2d98855c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:42.260423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d06fa1-6e9e-45f4-b5e1-2b2d98855c76 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:42.260703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d06fa1-6e9e-45f4-b5e1-2b2d98855c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:42.260975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d06fa1-6e9e-45f4-b5e1-2b2d98855c76 None admin] PATH_INFO: `/v3/users/ba48f9a366cb43dc8beca6198343da50` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:42.265022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be58d7-4ef7-4d9e-ba51-fff6fca727a3 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:42.274598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d06fa1-6e9e-45f4-b5e1-2b2d98855c76 None admin] RBAC: Authorizing `identity:get_user(user_id=ba48f9a366cb43dc8beca6198343da50)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:42.276067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d06fa1-6e9e-45f4-b5e1-2b2d98855c76 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:42.279882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 158/308] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:42 2022] GET /identity/v3/users/ba48f9a366cb43dc8beca6198343da50 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:42.279882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d06fa1-6e9e-45f4-b5e1-2b2d98855c76 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:42.305379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57411026-26f0-4131-a3e4-4e2ca317f49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:42.306078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57411026-26f0-4131-a3e4-4e2ca317f49f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:42.306355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57411026-26f0-4131-a3e4-4e2ca317f49f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:42.306887 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57411026-26f0-4131-a3e4-4e2ca317f49f None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:42.307620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57411026-26f0-4131-a3e4-4e2ca317f49f None admin] RBAC: Adding query filter params (name=member) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:42.307974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57411026-26f0-4131-a3e4-4e2ca317f49f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:42.309397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57411026-26f0-4131-a3e4-4e2ca317f49f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:42.317349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:42.318891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 151/309] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:42.319508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57411026-26f0-4131-a3e4-4e2ca317f49f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:42.339067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:42.396095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d06fa1-6e9e-45f4-b5e1-2b2d98855c76 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:42.430116 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57411026-26f0-4131-a3e4-4e2ca317f49f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:42.431888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e207a50-1476-498f-abda-984b9780a48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:42.432521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e207a50-1476-498f-abda-984b9780a48e None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:42.432801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e207a50-1476-498f-abda-984b9780a48e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:42.433071 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e207a50-1476-498f-abda-984b9780a48e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:42.444841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e207a50-1476-498f-abda-984b9780a48e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-4e207a50-1476-498f-abda-984b9780a48e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:42.449741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:42.450645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 159/310] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:42.452659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e207a50-1476-498f-abda-984b9780a48e None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:42.471669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730d5f4-ca47-4264-b3b4-6a2264344214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:42.472316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730d5f4-ca47-4264-b3b4-6a2264344214 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:42.472592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730d5f4-ca47-4264-b3b4-6a2264344214 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:42.472865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730d5f4-ca47-4264-b3b4-6a2264344214 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:42.473577 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730d5f4-ca47-4264-b3b4-6a2264344214 None admin] RBAC: Adding query filter params (role.id=4f0b83503c8847079d53d3dd089d98cb, user.id=ba48f9a366cb43dc8beca6198343da50, scope.system=all) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:42.473880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730d5f4-ca47-4264-b3b4-6a2264344214 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:42.475361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d730d5f4-ca47-4264-b3b4-6a2264344214 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:42.480420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 152/311] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Sep 7 16:03:42 2022] GET /identity/v3/role_assignments?user.id=ba48f9a366cb43dc8beca6198343da50&scope.system=all&role.id=4f0b83503c8847079d53d3dd089d98cb => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:42.481245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730d5f4-ca47-4264-b3b4-6a2264344214 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:42.577831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:42.585202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:42.598390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e207a50-1476-498f-abda-984b9780a48e None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:42.604325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d730d5f4-ca47-4264-b3b4-6a2264344214 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:42.649116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d163e-e02b-43ea-a81a-38f0ebc9f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:42.650275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d163e-e02b-43ea-a81a-38f0ebc9f32c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:42.650898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d163e-e02b-43ea-a81a-38f0ebc9f32c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:42.651504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6d163e-e02b-43ea-a81a-38f0ebc9f32c None admin] PATH_INFO: `/v3/users/2950b27761714033950e5954dfce0524` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:42.652711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1cebd-a9da-4b58-a9af-b6ca2682e349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:42.653146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d163e-e02b-43ea-a81a-38f0ebc9f32c None admin] RBAC: Authorizing `identity:get_user(user_id=2950b27761714033950e5954dfce0524)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:42.653485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1cebd-a9da-4b58-a9af-b6ca2682e349 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:42.653835 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1cebd-a9da-4b58-a9af-b6ca2682e349 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:42.654179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d1cebd-a9da-4b58-a9af-b6ca2682e349 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:42.654929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1cebd-a9da-4b58-a9af-b6ca2682e349 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:42.655300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1cebd-a9da-4b58-a9af-b6ca2682e349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:42.655743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6d163e-e02b-43ea-a81a-38f0ebc9f32c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:42.656935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d1cebd-a9da-4b58-a9af-b6ca2682e349 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:42.662746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 160/312] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:42 2022] GET /identity/v3/users/2950b27761714033950e5954dfce0524 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:42.664014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d163e-e02b-43ea-a81a-38f0ebc9f32c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:42.669118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 153/313] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:42.679169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1cebd-a9da-4b58-a9af-b6ca2682e349 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:42.776052 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6d163e-e02b-43ea-a81a-38f0ebc9f32c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:42.813042 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7cfdd-a926-493c-8127-cf219ce31cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:42.814034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7cfdd-a926-493c-8127-cf219ce31cc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:42.814369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d1cebd-a9da-4b58-a9af-b6ca2682e349 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:42.814751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7cfdd-a926-493c-8127-cf219ce31cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:42.815256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7cfdd-a926-493c-8127-cf219ce31cc2 None admin] PATH_INFO: `/v3/system/users/2950b27761714033950e5954dfce0524/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:42.816930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7cfdd-a926-493c-8127-cf219ce31cc2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2950b27761714033950e5954dfce0524, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:42.818490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7cfdd-a926-493c-8127-cf219ce31cc2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:42.850816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b7f65-0f0b-473a-ab16-399de40151eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:42.851377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b7f65-0f0b-473a-ab16-399de40151eb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:42.851717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b7f65-0f0b-473a-ab16-399de40151eb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:42.852017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b7f65-0f0b-473a-ab16-399de40151eb None admin] PATH_INFO: `/v3/users/82d7cb6b93ed4ee4b70e061f7bf56874` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:42.853272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b7f65-0f0b-473a-ab16-399de40151eb None admin] RBAC: Authorizing `identity:get_user(user_id=82d7cb6b93ed4ee4b70e061f7bf56874)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:42.854715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b7f65-0f0b-473a-ab16-399de40151eb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:42.856514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 154/314] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:42 2022] GET /identity/v3/users/82d7cb6b93ed4ee4b70e061f7bf56874 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:42.862700 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b7f65-0f0b-473a-ab16-399de40151eb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:42.872363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:42.936329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 161/315] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Sep 7 16:03:42 2022] PUT /identity/v3/system/users/2950b27761714033950e5954dfce0524/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:42.973513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b7f65-0f0b-473a-ab16-399de40151eb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:43.026542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d90848-e60a-4dbf-8816-2e9c04963812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:43.026542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90848-e60a-4dbf-8816-2e9c04963812 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:43.026542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90848-e60a-4dbf-8816-2e9c04963812 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:43.026542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90848-e60a-4dbf-8816-2e9c04963812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:43.107265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d90848-e60a-4dbf-8816-2e9c04963812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:43.109082 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d90848-e60a-4dbf-8816-2e9c04963812 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:43.118138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:43.217830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 155/316] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:43.223539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7cfdd-a926-493c-8127-cf219ce31cc2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:43.249658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:43.328585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7cfdd-a926-493c-8127-cf219ce31cc2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:43.364991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d6d45-a7ad-44b0-b82d-26749e84c225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:43.365845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d6d45-a7ad-44b0-b82d-26749e84c225 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:43.366295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d6d45-a7ad-44b0-b82d-26749e84c225 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:43.366711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d6d45-a7ad-44b0-b82d-26749e84c225 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:43.368000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d6d45-a7ad-44b0-b82d-26749e84c225 None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:43.369568 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d6d45-a7ad-44b0-b82d-26749e84c225 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:43.371617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 162/317] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:43 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:43.378157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d90848-e60a-4dbf-8816-2e9c04963812 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:43.482604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d90848-e60a-4dbf-8816-2e9c04963812 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:43.496102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:43.520499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc80c1-b7b4-4725-938b-a193cf23b865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:43.521110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc80c1-b7b4-4725-938b-a193cf23b865 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:43.521428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc80c1-b7b4-4725-938b-a193cf23b865 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:43.521705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc80c1-b7b4-4725-938b-a193cf23b865 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:43.522427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc80c1-b7b4-4725-938b-a193cf23b865 None admin] RBAC: Adding query filter params (role.id=4f0b83503c8847079d53d3dd089d98cb, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4, user.id=82d7cb6b93ed4ee4b70e061f7bf56874) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:43.522731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc80c1-b7b4-4725-938b-a193cf23b865 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:43.524180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc80c1-b7b4-4725-938b-a193cf23b865 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:43.530273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 156/318] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:03:43 2022] GET /identity/v3/role_assignments?user.id=82d7cb6b93ed4ee4b70e061f7bf56874&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=4f0b83503c8847079d53d3dd089d98cb => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:44.052702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea8073-4583-43a4-bef8-4497f9fa8960 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:44.053054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea8073-4583-43a4-bef8-4497f9fa8960 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:44.053287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea8073-4583-43a4-bef8-4497f9fa8960 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:44.054648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 163/319] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:44.060114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c29e6-0459-4019-9c18-3505c004455d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:44.060400 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c29e6-0459-4019-9c18-3505c004455d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:44.060673 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c29e6-0459-4019-9c18-3505c004455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:44.079258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-191c29e6-0459-4019-9c18-3505c004455d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:44.080466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-191c29e6-0459-4019-9c18-3505c004455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:44.146447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 157/320] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:44.412118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea8073-4583-43a4-bef8-4497f9fa8960 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:44.504680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:44.515410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea8073-4583-43a4-bef8-4497f9fa8960 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:44.548450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4c534-8c36-418a-97f8-e23edb44a602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:44.549087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4c534-8c36-418a-97f8-e23edb44a602 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:44.549897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4c534-8c36-418a-97f8-e23edb44a602 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:44.550166 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4c534-8c36-418a-97f8-e23edb44a602 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:44.551257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4c534-8c36-418a-97f8-e23edb44a602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:44.553096 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4c534-8c36-418a-97f8-e23edb44a602 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:44.555876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 164/321] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:03:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:44.560659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c29e6-0459-4019-9c18-3505c004455d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:44.660687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:44.664381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c29e6-0459-4019-9c18-3505c004455d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:44.700566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ff4da-e78a-4a0d-98ba-53a0d5735dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:44.701438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ff4da-e78a-4a0d-98ba-53a0d5735dab None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:44.701863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ff4da-e78a-4a0d-98ba-53a0d5735dab None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:44.702277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ff4da-e78a-4a0d-98ba-53a0d5735dab None admin] PATH_INFO: `/v3/groups` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:44.703056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ff4da-e78a-4a0d-98ba-53a0d5735dab None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:44.704668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ff4da-e78a-4a0d-98ba-53a0d5735dab None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:44.712511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e33789-fc94-40dc-aa56-9e636b8f5958 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:44.712795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e33789-fc94-40dc-aa56-9e636b8f5958 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:44.713071 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e33789-fc94-40dc-aa56-9e636b8f5958 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:44.714389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 165/322] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:03:44.719127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e640cd1-b77b-442e-a313-7e9a7a745b70 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:44.719522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e640cd1-b77b-442e-a313-7e9a7a745b70 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:44.719819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e640cd1-b77b-442e-a313-7e9a7a745b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:44.739077 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-4e640cd1-b77b-442e-a313-7e9a7a745b70 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:44.740288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4e640cd1-b77b-442e-a313-7e9a7a745b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:44.807381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 166/323] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:44.816037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 158/324] 10.222.0.15 () {60 vars in 1134 bytes} [Wed Sep 7 16:03:44 2022] POST /identity/v3/groups => generated 226 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:03:45.073959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ff4da-e78a-4a0d-98ba-53a0d5735dab None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:45.173913 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0d747-84d9-425d-90ae-39882be6a41f None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:45.174296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0d747-84d9-425d-90ae-39882be6a41f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:45.174564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0d747-84d9-425d-90ae-39882be6a41f None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:45.176192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 167/325] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:45.180970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ff4da-e78a-4a0d-98ba-53a0d5735dab None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:45.181752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be558def-c998-4582-840c-4b112ba62c3b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:45.182038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be558def-c998-4582-840c-4b112ba62c3b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:45.182315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be558def-c998-4582-840c-4b112ba62c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:45.203492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-be558def-c998-4582-840c-4b112ba62c3b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:45.204725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-be558def-c998-4582-840c-4b112ba62c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:45.216376 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c19a-33dc-4254-95fc-a1f2da97ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:45.217006 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c19a-33dc-4254-95fc-a1f2da97ac56 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:45.217279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c19a-33dc-4254-95fc-a1f2da97ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:45.218023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c19a-33dc-4254-95fc-a1f2da97ac56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:45.231426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9c19a-33dc-4254-95fc-a1f2da97ac56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-3af9c19a-33dc-4254-95fc-a1f2da97ac56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:45.237168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:45.238048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:45.238820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 159/326] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:45.244334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c19a-33dc-4254-95fc-a1f2da97ac56 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:45.269706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:45.294786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 168/327] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:45.360690 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c19a-33dc-4254-95fc-a1f2da97ac56 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:45.400357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae715fa3-8660-4de6-a43e-94c0ecd4e6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:45.400976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae715fa3-8660-4de6-a43e-94c0ecd4e6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:45.401252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae715fa3-8660-4de6-a43e-94c0ecd4e6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:45.401523 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae715fa3-8660-4de6-a43e-94c0ecd4e6b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:45.402186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae715fa3-8660-4de6-a43e-94c0ecd4e6b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:45.402485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae715fa3-8660-4de6-a43e-94c0ecd4e6b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:45.404419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae715fa3-8660-4de6-a43e-94c0ecd4e6b1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:45.414076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 160/328] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:45.419488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be558def-c998-4582-840c-4b112ba62c3b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:45.443661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:45.533672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be558def-c998-4582-840c-4b112ba62c3b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:45.570280 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5e367-10f7-4569-a555-3267c7a8f1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:45.571287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5e367-10f7-4569-a555-3267c7a8f1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:45.571780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5e367-10f7-4569-a555-3267c7a8f1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:45.572286 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5e367-10f7-4569-a555-3267c7a8f1d4 None admin] PATH_INFO: `/v3/users/ba48f9a366cb43dc8beca6198343da50` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:45.573676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5e367-10f7-4569-a555-3267c7a8f1d4 None admin] RBAC: Authorizing `identity:get_user(user_id=ba48f9a366cb43dc8beca6198343da50)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:45.575304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5e367-10f7-4569-a555-3267c7a8f1d4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:45.577485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 169/329] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:45 2022] GET /identity/v3/users/ba48f9a366cb43dc8beca6198343da50 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:45.582413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae715fa3-8660-4de6-a43e-94c0ecd4e6b1 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:45.584050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5e367-10f7-4569-a555-3267c7a8f1d4 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:45.693509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:45.704035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:45.720781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae715fa3-8660-4de6-a43e-94c0ecd4e6b1 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:45.727780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5e367-10f7-4569-a555-3267c7a8f1d4 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:45.764516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c032b77-e11d-423d-9d7d-23cf88f8a38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:45.765411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c032b77-e11d-423d-9d7d-23cf88f8a38e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:45.765842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c032b77-e11d-423d-9d7d-23cf88f8a38e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:45.766322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c032b77-e11d-423d-9d7d-23cf88f8a38e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:45.779410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b870a-799d-49ff-be1e-1d6c3dcc0faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:45.780101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b870a-799d-49ff-be1e-1d6c3dcc0faf None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:45.780382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b870a-799d-49ff-be1e-1d6c3dcc0faf None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:45.780640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c032b77-e11d-423d-9d7d-23cf88f8a38e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:45.780755 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b870a-799d-49ff-be1e-1d6c3dcc0faf None admin] PATH_INFO: `/v3/system/users/ba48f9a366cb43dc8beca6198343da50/roles/4f0b83503c8847079d53d3dd089d98cb` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:45.782074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b870a-799d-49ff-be1e-1d6c3dcc0faf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ba48f9a366cb43dc8beca6198343da50, role_id=4f0b83503c8847079d53d3dd089d98cb)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:45.783448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b870a-799d-49ff-be1e-1d6c3dcc0faf None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-5c032b77-e11d-423d-9d7d-23cf88f8a38e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:45.788732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:45.789630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:45.790412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:45.793035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 161/330] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:45.798551 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c032b77-e11d-423d-9d7d-23cf88f8a38e None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:45.849284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 170/331] 10.222.0.15 () {58 vars in 1334 bytes} [Wed Sep 7 16:03:45 2022] PUT /identity/v3/system/users/ba48f9a366cb43dc8beca6198343da50/roles/4f0b83503c8847079d53d3dd089d98cb => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:45.920891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c032b77-e11d-423d-9d7d-23cf88f8a38e None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:45.977993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e8510-3423-4ea5-b77d-eafa962f9bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:45.978619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e8510-3423-4ea5-b77d-eafa962f9bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:45.978899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e8510-3423-4ea5-b77d-eafa962f9bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:45.979170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e8510-3423-4ea5-b77d-eafa962f9bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:45.979875 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e8510-3423-4ea5-b77d-eafa962f9bb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:45.980180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e8510-3423-4ea5-b77d-eafa962f9bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:45.981588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e8510-3423-4ea5-b77d-eafa962f9bb4 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:45.992444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 162/332] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:45.998631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b870a-799d-49ff-be1e-1d6c3dcc0faf None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:46.131756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b870a-799d-49ff-be1e-1d6c3dcc0faf None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:46.172904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309bdf2-8858-44bd-8b13-d843f0cf5716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:46.173547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309bdf2-8858-44bd-8b13-d843f0cf5716 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:46.173830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309bdf2-8858-44bd-8b13-d843f0cf5716 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:46.174129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309bdf2-8858-44bd-8b13-d843f0cf5716 None admin] PATH_INFO: `/v3/users/2950b27761714033950e5954dfce0524` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:46.175901 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309bdf2-8858-44bd-8b13-d843f0cf5716 None admin] RBAC: Authorizing `identity:get_user(user_id=2950b27761714033950e5954dfce0524)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:46.177361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309bdf2-8858-44bd-8b13-d843f0cf5716 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:46.179274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 171/333] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:45 2022] GET /identity/v3/users/2950b27761714033950e5954dfce0524 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:46.185207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e8510-3423-4ea5-b77d-eafa962f9bb4 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:46.190799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:46.334165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e8510-3423-4ea5-b77d-eafa962f9bb4 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:46.371886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8815a39-6ab7-4723-a958-d111e24139d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:46.372877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8815a39-6ab7-4723-a958-d111e24139d1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:46.373350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8815a39-6ab7-4723-a958-d111e24139d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:46.373816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8815a39-6ab7-4723-a958-d111e24139d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:46.374727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8815a39-6ab7-4723-a958-d111e24139d1 None admin] RBAC: Adding query filter params (role.id=f463de99ba174eab9396eef514e34e15, user.id=2950b27761714033950e5954dfce0524, scope.system=all) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:46.375225 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8815a39-6ab7-4723-a958-d111e24139d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:46.376891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8815a39-6ab7-4723-a958-d111e24139d1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:46.392212 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 163/334] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Sep 7 16:03:46 2022] GET /identity/v3/role_assignments?user.id=2950b27761714033950e5954dfce0524&scope.system=all&role.id=f463de99ba174eab9396eef514e34e15 => generated 510 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:48.124424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e024da-b17e-4b7d-8897-4926a14ba7f0 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:48.124950 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e024da-b17e-4b7d-8897-4926a14ba7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:48.124950 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e024da-b17e-4b7d-8897-4926a14ba7f0 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:48.126304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 172/335] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:48.141405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37409b-fb21-4688-9e6b-ecc302f7457e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:48.142330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37409b-fb21-4688-9e6b-ecc302f7457e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:48.142836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37409b-fb21-4688-9e6b-ecc302f7457e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:48.146592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f691c-f1db-4caf-a926-c1f8f529dd9d None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:48.147173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f691c-f1db-4caf-a926-c1f8f529dd9d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:48.147652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f691c-f1db-4caf-a926-c1f8f529dd9d None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:48.149549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 173/336] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:03:48.155272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9e58e-cd95-4f40-902c-5e669525da2e None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:48.155886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9e58e-cd95-4f40-902c-5e669525da2e None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:48.156364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9e58e-cd95-4f40-902c-5e669525da2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:48.166932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-6b37409b-fb21-4688-9e6b-ecc302f7457e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:48.168514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6b37409b-fb21-4688-9e6b-ecc302f7457e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:48.192908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-84f9e58e-cd95-4f40-902c-5e669525da2e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:48.194543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-84f9e58e-cd95-4f40-902c-5e669525da2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:48.225196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:48.257005 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:48.276474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 164/337] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:48.335342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 174/338] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:48.600905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37409b-fb21-4688-9e6b-ecc302f7457e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:48.737162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37409b-fb21-4688-9e6b-ecc302f7457e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:48.777549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfe4f4-72cc-48b6-94c0-bbd597255566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:48.778182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe4f4-72cc-48b6-94c0-bbd597255566 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:48.778457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe4f4-72cc-48b6-94c0-bbd597255566 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:48.778725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfe4f4-72cc-48b6-94c0-bbd597255566 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:48.791925 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfe4f4-72cc-48b6-94c0-bbd597255566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-adbfe4f4-72cc-48b6-94c0-bbd597255566 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:48.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:48.797629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:48.798411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 165/339] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:48.805240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9e58e-cd95-4f40-902c-5e669525da2e None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:48.816994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd24da0-bf4d-4956-b5f7-66f21752a899 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:48.817273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd24da0-bf4d-4956-b5f7-66f21752a899 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:48.817552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd24da0-bf4d-4956-b5f7-66f21752a899 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:48.823831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 166/340] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:03:48.840918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f371-3509-4a13-b7ec-b7d004dd50ae None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:48.841255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f371-3509-4a13-b7ec-b7d004dd50ae None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:48.841467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f371-3509-4a13-b7ec-b7d004dd50ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:48.873899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-4ae1f371-3509-4a13-b7ec-b7d004dd50ae None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:48.875292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4ae1f371-3509-4a13-b7ec-b7d004dd50ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:48.928471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:48.952291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 167/341] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:48.953535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a1243-fa23-42ac-85cf-3d8fbaa71065 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:48.953817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a1243-fa23-42ac-85cf-3d8fbaa71065 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:48.954086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a1243-fa23-42ac-85cf-3d8fbaa71065 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:48.955821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9e58e-cd95-4f40-902c-5e669525da2e None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:48.960233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 168/342] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:48.961265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a1243-fa23-42ac-85cf-3d8fbaa71065 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:48.999670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558ffc4-81be-4a50-bf60-381bbe49c6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:49.000722 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558ffc4-81be-4a50-bf60-381bbe49c6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:49.001205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558ffc4-81be-4a50-bf60-381bbe49c6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:49.001677 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558ffc4-81be-4a50-bf60-381bbe49c6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:49.002642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558ffc4-81be-4a50-bf60-381bbe49c6e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:49.003145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558ffc4-81be-4a50-bf60-381bbe49c6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:49.005219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558ffc4-81be-4a50-bf60-381bbe49c6e9 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:49.015979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 175/343] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:49.018503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c328350-26ff-4e65-9c4c-8f42af297162 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:49.018991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c328350-26ff-4e65-9c4c-8f42af297162 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:49.024553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c328350-26ff-4e65-9c4c-8f42af297162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:49.045933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-2c328350-26ff-4e65-9c4c-8f42af297162 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:49.047605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2c328350-26ff-4e65-9c4c-8f42af297162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:49.050984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:49.129918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 176/344] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:49.130500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c328350-26ff-4e65-9c4c-8f42af297162 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:49.208450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a1243-fa23-42ac-85cf-3d8fbaa71065 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:49.257004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c328350-26ff-4e65-9c4c-8f42af297162 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:49.261214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c324a-befa-4d3c-a5cf-f87afe7bb9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:49.262054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c324a-befa-4d3c-a5cf-f87afe7bb9fe None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:49.262482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c324a-befa-4d3c-a5cf-f87afe7bb9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:49.262891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c324a-befa-4d3c-a5cf-f87afe7bb9fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:49.277814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c324a-befa-4d3c-a5cf-f87afe7bb9fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-739c324a-befa-4d3c-a5cf-f87afe7bb9fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:49.283326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:49.284258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:49.285046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:49.287629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 169/345] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:03:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:49.293603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c324a-befa-4d3c-a5cf-f87afe7bb9fe None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:49.298221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9cc3e-0f09-4d97-875a-a7869b5bdc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:49.298385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9cc3e-0f09-4d97-875a-a7869b5bdc13 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:49.298793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9cc3e-0f09-4d97-875a-a7869b5bdc13 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:49.299163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9cc3e-0f09-4d97-875a-a7869b5bdc13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:49.319790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:49.355792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:49.383275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9cc3e-0f09-4d97-875a-a7869b5bdc13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:49.383275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9cc3e-0f09-4d97-875a-a7869b5bdc13 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:49.420630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c324a-befa-4d3c-a5cf-f87afe7bb9fe None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:49.459504 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6c4f9-2928-4641-9f60-d585453f2e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:49.460429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6c4f9-2928-4641-9f60-d585453f2e90 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:49.460876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6c4f9-2928-4641-9f60-d585453f2e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:49.461342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6c4f9-2928-4641-9f60-d585453f2e90 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:49.462171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6c4f9-2928-4641-9f60-d585453f2e90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:49.462671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6c4f9-2928-4641-9f60-d585453f2e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:49.465674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6c4f9-2928-4641-9f60-d585453f2e90 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:49.474942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 170/346] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:03:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:49.480719 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6c4f9-2928-4641-9f60-d585453f2e90 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:49.499723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 177/347] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:49.506183 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9cc3e-0f09-4d97-875a-a7869b5bdc13 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:49.560174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:49.597751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6c4f9-2928-4641-9f60-d585453f2e90 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:49.600446 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:49.632207 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9cc3e-0f09-4d97-875a-a7869b5bdc13 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:49.647986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f013775-01ee-4641-b486-fdf39dfa58e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:49.649952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f013775-01ee-4641-b486-fdf39dfa58e9 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:49.650625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f013775-01ee-4641-b486-fdf39dfa58e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:49.651747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f013775-01ee-4641-b486-fdf39dfa58e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:49.695928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9deff12-2761-454e-9702-439e125f9dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:49.696946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9deff12-2761-454e-9702-439e125f9dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:49.697417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9deff12-2761-454e-9702-439e125f9dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:49.697879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9deff12-2761-454e-9702-439e125f9dc3 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:49.699336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9deff12-2761-454e-9702-439e125f9dc3 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:49.701087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9deff12-2761-454e-9702-439e125f9dc3 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:49.703372 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 178/348] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:49 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:49.704431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9deff12-2761-454e-9702-439e125f9dc3 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:49.777902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f013775-01ee-4641-b486-fdf39dfa58e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:49.779386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f013775-01ee-4641-b486-fdf39dfa58e9 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:49.837809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9deff12-2761-454e-9702-439e125f9dc3 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:49.851188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:49.877911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5842bfe-f54a-46a5-b291-84d4d9b276bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:49.878555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5842bfe-f54a-46a5-b291-84d4d9b276bf None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:49.883577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5842bfe-f54a-46a5-b291-84d4d9b276bf None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:49.883577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5842bfe-f54a-46a5-b291-84d4d9b276bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:49.893232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5842bfe-f54a-46a5-b291-84d4d9b276bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-a5842bfe-f54a-46a5-b291-84d4d9b276bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:49.898621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:49.899547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 16:03:49.900393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:49.902744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 179/349] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:49.903555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5842bfe-f54a-46a5-b291-84d4d9b276bf None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:49.910827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 171/350] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:49.911775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f013775-01ee-4641-b486-fdf39dfa58e9 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:49.923140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:50.049968 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5842bfe-f54a-46a5-b291-84d4d9b276bf None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:50.058100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f013775-01ee-4641-b486-fdf39dfa58e9 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:50.092894 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224303a4-95ee-4aa7-96dd-96da8471693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:50.093681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224303a4-95ee-4aa7-96dd-96da8471693f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.094089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224303a4-95ee-4aa7-96dd-96da8471693f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.094462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224303a4-95ee-4aa7-96dd-96da8471693f None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.102493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224303a4-95ee-4aa7-96dd-96da8471693f None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:50.104124 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224303a4-95ee-4aa7-96dd-96da8471693f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:50.106177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 180/351] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:03:49 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:50.107595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed929c16-5164-4d8f-bd67-3d61f1959d8d None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.107906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed929c16-5164-4d8f-bd67-3d61f1959d8d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.108182 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed929c16-5164-4d8f-bd67-3d61f1959d8d None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.109386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 181/352] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:50.114117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078bf00-e365-4f71-a9fe-2623e387f471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:50.114713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed929c16-5164-4d8f-bd67-3d61f1959d8d None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:50.114814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078bf00-e365-4f71-a9fe-2623e387f471 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.115091 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078bf00-e365-4f71-a9fe-2623e387f471 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.115366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078bf00-e365-4f71-a9fe-2623e387f471 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.127986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078bf00-e365-4f71-a9fe-2623e387f471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-2078bf00-e365-4f71-a9fe-2623e387f471 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:50.133181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:50.134304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:50.135211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 172/353] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:50.135973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078bf00-e365-4f71-a9fe-2623e387f471 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:50.162443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:50.222176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:50.270214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078bf00-e365-4f71-a9fe-2623e387f471 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:50.271847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed929c16-5164-4d8f-bd67-3d61f1959d8d None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:50.312235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd55333-0a10-4257-b055-a57b932c8eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:50.312884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd55333-0a10-4257-b055-a57b932c8eab None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.313163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd55333-0a10-4257-b055-a57b932c8eab None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.313440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd55333-0a10-4257-b055-a57b932c8eab None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.314252 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3033ee30-7f2c-4039-a0b8-37ffd1798916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:50.314907 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033ee30-7f2c-4039-a0b8-37ffd1798916 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.315188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033ee30-7f2c-4039-a0b8-37ffd1798916 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.315401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd55333-0a10-4257-b055-a57b932c8eab None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:50.315499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3033ee30-7f2c-4039-a0b8-37ffd1798916 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.316174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033ee30-7f2c-4039-a0b8-37ffd1798916 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:50.316491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033ee30-7f2c-4039-a0b8-37ffd1798916 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:50.316939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd55333-0a10-4257-b055-a57b932c8eab None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:50.317826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3033ee30-7f2c-4039-a0b8-37ffd1798916 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:50.318608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 173/354] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:50 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:50.319370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd55333-0a10-4257-b055-a57b932c8eab None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:50.328159 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 182/355] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:50.334387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74bc8d-02b1-48ec-9b3d-f999d34c1c12 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.334648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74bc8d-02b1-48ec-9b3d-f999d34c1c12 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.334926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74bc8d-02b1-48ec-9b3d-f999d34c1c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.361520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-2a74bc8d-02b1-48ec-9b3d-f999d34c1c12 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:50.362761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2a74bc8d-02b1-48ec-9b3d-f999d34c1c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:50.436966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 183/356] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:50.438107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74bc8d-02b1-48ec-9b3d-f999d34c1c12 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:50.448702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd55333-0a10-4257-b055-a57b932c8eab None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:50.467983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:50.496279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:50.504717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcea70f-ad6b-400e-9aea-d3787596c1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:50.505323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcea70f-ad6b-400e-9aea-d3787596c1fa None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.505601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcea70f-ad6b-400e-9aea-d3787596c1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.505869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcea70f-ad6b-400e-9aea-d3787596c1fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.506594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcea70f-ad6b-400e-9aea-d3787596c1fa None admin] RBAC: Adding query filter params (group.id=c3893fedc58b4b7fa399c12d2c3afcb6, role.id=f463de99ba174eab9396eef514e34e15, scope.project.id=53b19d145ddf4f3c950098b6269b33e0) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:50.506896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcea70f-ad6b-400e-9aea-d3787596c1fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:50.508354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcea70f-ad6b-400e-9aea-d3787596c1fa None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:50.515733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 174/357] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 7 16:03:50 2022] GET /identity/v3/role_assignments?group.id=c3893fedc58b4b7fa399c12d2c3afcb6&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=f463de99ba174eab9396eef514e34e15 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:50.517179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771a2e-873e-48d2-8269-425c9b31db05 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.517461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771a2e-873e-48d2-8269-425c9b31db05 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.518315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e771a2e-873e-48d2-8269-425c9b31db05 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.519732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 175/358] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:50.521878 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e771a2e-873e-48d2-8269-425c9b31db05 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:50.565703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74bc8d-02b1-48ec-9b3d-f999d34c1c12 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:50.621493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c81da3-a9cf-40cb-97e2-45ea7eb8012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:50.622546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c81da3-a9cf-40cb-97e2-45ea7eb8012c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.623322 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c81da3-a9cf-40cb-97e2-45ea7eb8012c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.624055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c81da3-a9cf-40cb-97e2-45ea7eb8012c None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.624915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c81da3-a9cf-40cb-97e2-45ea7eb8012c None admin] RBAC: Adding query filter params (name=member) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:50.625377 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c81da3-a9cf-40cb-97e2-45ea7eb8012c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:50.627668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c81da3-a9cf-40cb-97e2-45ea7eb8012c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:50.640305 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 184/359] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:50.641451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c81da3-a9cf-40cb-97e2-45ea7eb8012c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:50.677867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e771a2e-873e-48d2-8269-425c9b31db05 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:50.735655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0a363-2e04-4d17-8f8f-e240e5ce8f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:50.735655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0a363-2e04-4d17-8f8f-e240e5ce8f86 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.735655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0a363-2e04-4d17-8f8f-e240e5ce8f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.735655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0a363-2e04-4d17-8f8f-e240e5ce8f86 None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.736219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0a363-2e04-4d17-8f8f-e240e5ce8f86 None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:50.736219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0a363-2e04-4d17-8f8f-e240e5ce8f86 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:50.736219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 176/360] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:03:50 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:50.738011 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91023bb5-4654-4da7-8f9b-fa02a6d8082e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.738294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91023bb5-4654-4da7-8f9b-fa02a6d8082e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.738566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91023bb5-4654-4da7-8f9b-fa02a6d8082e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.759522 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-91023bb5-4654-4da7-8f9b-fa02a6d8082e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:50.760708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-91023bb5-4654-4da7-8f9b-fa02a6d8082e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:50.768564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:50.786495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c81da3-a9cf-40cb-97e2-45ea7eb8012c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:50.834392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:50.842424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2a473-d39e-41d6-8d23-74182c300458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:50.843071 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2a473-d39e-41d6-8d23-74182c300458 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:50.843507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 177/361] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:50.843614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2a473-d39e-41d6-8d23-74182c300458 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:50.843710 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2a473-d39e-41d6-8d23-74182c300458 None admin] PATH_INFO: `/v3/users/ba48f9a366cb43dc8beca6198343da50` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:50.847860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91023bb5-4654-4da7-8f9b-fa02a6d8082e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:50.848636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2a473-d39e-41d6-8d23-74182c300458 None admin] RBAC: Authorizing `identity:get_user(user_id=ba48f9a366cb43dc8beca6198343da50)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:50.850102 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2a473-d39e-41d6-8d23-74182c300458 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:50.851876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 185/362] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:03:50 2022] GET /identity/v3/users/ba48f9a366cb43dc8beca6198343da50 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:50.852711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2a473-d39e-41d6-8d23-74182c300458 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:50.982978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91023bb5-4654-4da7-8f9b-fa02a6d8082e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:50.992277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2a473-d39e-41d6-8d23-74182c300458 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:51.021793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d3c2a-4d38-426d-836c-62d47770be48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:51.022462 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d3c2a-4d38-426d-836c-62d47770be48 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:51.022744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d3c2a-4d38-426d-836c-62d47770be48 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:51.023210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d3c2a-4d38-426d-836c-62d47770be48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:51.028608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636fb1e-f120-4900-a2f7-1ed2e209dad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:51.029283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636fb1e-f120-4900-a2f7-1ed2e209dad3 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:51.029561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636fb1e-f120-4900-a2f7-1ed2e209dad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:51.029839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636fb1e-f120-4900-a2f7-1ed2e209dad3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:51.041947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636fb1e-f120-4900-a2f7-1ed2e209dad3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-3636fb1e-f120-4900-a2f7-1ed2e209dad3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:51.046970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:51.047882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:51.048698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 186/363] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:03:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:51.050812 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636fb1e-f120-4900-a2f7-1ed2e209dad3 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:51.067107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:51.093843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:51.102204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d3c2a-4d38-426d-836c-62d47770be48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:51.103675 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d3c2a-4d38-426d-836c-62d47770be48 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:51.169108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636fb1e-f120-4900-a2f7-1ed2e209dad3 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:51.212391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749c071-9325-4604-9ae2-ae73923d4b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:51.213007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749c071-9325-4604-9ae2-ae73923d4b9b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:51.213967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749c071-9325-4604-9ae2-ae73923d4b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:51.214253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749c071-9325-4604-9ae2-ae73923d4b9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:51.214986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749c071-9325-4604-9ae2-ae73923d4b9b None admin] RBAC: Adding query filter params (group.id=c3893fedc58b4b7fa399c12d2c3afcb6, role.id=f8c4be33fa2f4e69a3de71a3c4d7b179, scope.project.id=53b19d145ddf4f3c950098b6269b33e0) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:51.216055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749c071-9325-4604-9ae2-ae73923d4b9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:51.216769 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749c071-9325-4604-9ae2-ae73923d4b9b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:51.223402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 187/364] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 7 16:03:51 2022] GET /identity/v3/role_assignments?group.id=c3893fedc58b4b7fa399c12d2c3afcb6&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:51.224066 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749c071-9325-4604-9ae2-ae73923d4b9b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:51.228968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 178/365] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:51.229929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d3c2a-4d38-426d-836c-62d47770be48 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:51.323181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:51.353370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749c071-9325-4604-9ae2-ae73923d4b9b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:51.367479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:51.375117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d3c2a-4d38-426d-836c-62d47770be48 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:51.400698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1ae91-9ea3-481f-8bd8-36801bec7498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:51.401573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1ae91-9ea3-481f-8bd8-36801bec7498 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:51.401995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1ae91-9ea3-481f-8bd8-36801bec7498 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:51.402432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1ae91-9ea3-481f-8bd8-36801bec7498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:51.403380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda1ae91-9ea3-481f-8bd8-36801bec7498 None admin] RBAC: Adding query filter params (role.id=4f0b83503c8847079d53d3dd089d98cb, user.id=ba48f9a366cb43dc8beca6198343da50, scope.system=all) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:51.404225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda1ae91-9ea3-481f-8bd8-36801bec7498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:51.407688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda1ae91-9ea3-481f-8bd8-36801bec7498 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:51.434323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 188/366] 10.222.0.15 () {56 vars in 1308 bytes} [Wed Sep 7 16:03:51 2022] GET /identity/v3/role_assignments?user.id=ba48f9a366cb43dc8beca6198343da50&scope.system=all&role.id=4f0b83503c8847079d53d3dd089d98cb => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:51.435015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1ae91-9ea3-481f-8bd8-36801bec7498 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:51.448904 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f311557-2a5a-454b-87b9-ff44e03cb4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:51.449543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311557-2a5a-454b-87b9-ff44e03cb4de None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:51.449830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311557-2a5a-454b-87b9-ff44e03cb4de None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:51.450104 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f311557-2a5a-454b-87b9-ff44e03cb4de None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:51.450795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f311557-2a5a-454b-87b9-ff44e03cb4de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:51.451099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f311557-2a5a-454b-87b9-ff44e03cb4de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:51.452546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f311557-2a5a-454b-87b9-ff44e03cb4de None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:51.473318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 179/367] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:03:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:51.474257 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f311557-2a5a-454b-87b9-ff44e03cb4de None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:51.600658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1ae91-9ea3-481f-8bd8-36801bec7498 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:51.632866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f311557-2a5a-454b-87b9-ff44e03cb4de None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:51.644994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6257e6e-b516-422f-8913-da9353457885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:51.645793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6257e6e-b516-422f-8913-da9353457885 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:51.646082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6257e6e-b516-422f-8913-da9353457885 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:51.646367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6257e6e-b516-422f-8913-da9353457885 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:51.656199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:51.665087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6257e6e-b516-422f-8913-da9353457885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-b6257e6e-b516-422f-8913-da9353457885 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:51.670855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:51.671876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 189/368] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:03:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:51.673106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6257e6e-b516-422f-8913-da9353457885 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:51.694974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58337f00-f779-41a2-8e6a-f534869997fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:51.695638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58337f00-f779-41a2-8e6a-f534869997fb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:51.696001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58337f00-f779-41a2-8e6a-f534869997fb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:51.696284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58337f00-f779-41a2-8e6a-f534869997fb None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:51.698718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58337f00-f779-41a2-8e6a-f534869997fb None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:51.702514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58337f00-f779-41a2-8e6a-f534869997fb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:51.704552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 180/369] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:51 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:51.708550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58337f00-f779-41a2-8e6a-f534869997fb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:51.746229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:51.820105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6257e6e-b516-422f-8913-da9353457885 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:51.839844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58337f00-f779-41a2-8e6a-f534869997fb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:51.859848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464832e-8df8-4824-82be-b27c6f0566ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:51.860478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464832e-8df8-4824-82be-b27c6f0566ce None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:51.860761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464832e-8df8-4824-82be-b27c6f0566ce None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:51.861041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464832e-8df8-4824-82be-b27c6f0566ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:51.882741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6446c37-449b-49cb-9cfd-187487511a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:51.882741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6446c37-449b-49cb-9cfd-187487511a05 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:51.882741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6446c37-449b-49cb-9cfd-187487511a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:51.882741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6446c37-449b-49cb-9cfd-187487511a05 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:51.883555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6446c37-449b-49cb-9cfd-187487511a05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:51.883555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6446c37-449b-49cb-9cfd-187487511a05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:51.886508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6446c37-449b-49cb-9cfd-187487511a05 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:51.895921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 181/370] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:51.896505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6446c37-449b-49cb-9cfd-187487511a05 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:51.970060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:51.979196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464832e-8df8-4824-82be-b27c6f0566ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:51.980651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464832e-8df8-4824-82be-b27c6f0566ce None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:52.024679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:52.045170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6446c37-449b-49cb-9cfd-187487511a05 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:52.088857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b2e49-4cc3-4468-8154-52ffeda8a133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:52.089873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b2e49-4cc3-4468-8154-52ffeda8a133 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:52.090352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b2e49-4cc3-4468-8154-52ffeda8a133 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:52.090840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b2e49-4cc3-4468-8154-52ffeda8a133 None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:52.092371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b2e49-4cc3-4468-8154-52ffeda8a133 None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:52.094056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b2e49-4cc3-4468-8154-52ffeda8a133 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:52.102859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 182/371] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:03:51 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:52.104031 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b2e49-4cc3-4468-8154-52ffeda8a133 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:52.113695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 190/372] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:52.114493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464832e-8df8-4824-82be-b27c6f0566ce None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:52.226377 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:52.228086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b2e49-4cc3-4468-8154-52ffeda8a133 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:52.233328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464832e-8df8-4824-82be-b27c6f0566ce None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:52.270224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c96db4-a4ae-4d1c-b09a-bb241e753d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:52.272725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c96db4-a4ae-4d1c-b09a-bb241e753d9c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:52.273214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c96db4-a4ae-4d1c-b09a-bb241e753d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:52.273709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c96db4-a4ae-4d1c-b09a-bb241e753d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:52.278525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee278a-b094-4afd-8feb-722d807796ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:52.278953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee278a-b094-4afd-8feb-722d807796ed None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:52.279235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee278a-b094-4afd-8feb-722d807796ed None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:52.279698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee278a-b094-4afd-8feb-722d807796ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:52.280459 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee278a-b094-4afd-8feb-722d807796ed None admin] RBAC: Adding query filter params (group.id=c3893fedc58b4b7fa399c12d2c3afcb6, role.id=f463de99ba174eab9396eef514e34e15, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:52.280775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee278a-b094-4afd-8feb-722d807796ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:52.282235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ee278a-b094-4afd-8feb-722d807796ed None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:52.294909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 191/373] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 7 16:03:52 2022] GET /identity/v3/role_assignments?group.id=c3893fedc58b4b7fa399c12d2c3afcb6&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=f463de99ba174eab9396eef514e34e15 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:52.294909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee278a-b094-4afd-8feb-722d807796ed None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:52.319813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:52.381757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c96db4-a4ae-4d1c-b09a-bb241e753d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:52.384406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c96db4-a4ae-4d1c-b09a-bb241e753d9c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:52.424565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ee278a-b094-4afd-8feb-722d807796ed None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:52.474358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8251d60-f5a1-4425-b4d4-c4b961fb3298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:52.475073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8251d60-f5a1-4425-b4d4-c4b961fb3298 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:52.475357 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8251d60-f5a1-4425-b4d4-c4b961fb3298 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:52.475693 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8251d60-f5a1-4425-b4d4-c4b961fb3298 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:52.477667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8251d60-f5a1-4425-b4d4-c4b961fb3298 None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:52.479420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8251d60-f5a1-4425-b4d4-c4b961fb3298 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:52.485598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 192/374] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:52 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:52.490738 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8251d60-f5a1-4425-b4d4-c4b961fb3298 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:52.519402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 183/375] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:52.526714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c96db4-a4ae-4d1c-b09a-bb241e753d9c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:52.537196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:52.628220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:52.684969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8251d60-f5a1-4425-b4d4-c4b961fb3298 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:52.691135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c96db4-a4ae-4d1c-b09a-bb241e753d9c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:52.742670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7724ad-7ff3-4440-87bd-a58d09a0ff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:52.744344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7724ad-7ff3-4440-87bd-a58d09a0ff6b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:52.744846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7724ad-7ff3-4440-87bd-a58d09a0ff6b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:52.745471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7724ad-7ff3-4440-87bd-a58d09a0ff6b None admin] PATH_INFO: `/v3/projects/4f97db8bc2c2455d85720c36a1ed180d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:52.747420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7724ad-7ff3-4440-87bd-a58d09a0ff6b None admin] RBAC: Authorizing `identity:get_project(project_id=4f97db8bc2c2455d85720c36a1ed180d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:52.749607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7724ad-7ff3-4440-87bd-a58d09a0ff6b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:52.753282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 184/376] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:52 2022] GET /identity/v3/projects/4f97db8bc2c2455d85720c36a1ed180d => generated 344 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:52.759257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b85a9-4cc8-4901-aee8-b0e0494e9c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:52.759755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7724ad-7ff3-4440-87bd-a58d09a0ff6b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:52.760008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b85a9-4cc8-4901-aee8-b0e0494e9c0a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:52.760205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b85a9-4cc8-4901-aee8-b0e0494e9c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:52.760485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b85a9-4cc8-4901-aee8-b0e0494e9c0a None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:52.762381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b85a9-4cc8-4901-aee8-b0e0494e9c0a None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:52.763875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b85a9-4cc8-4901-aee8-b0e0494e9c0a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:52.765788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 193/377] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:03:52 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:52.772809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b85a9-4cc8-4901-aee8-b0e0494e9c0a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:52.869913 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7724ad-7ff3-4440-87bd-a58d09a0ff6b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:52.896179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:52.908975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea7d09-7a91-4ba6-a1c3-0feb18982080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:52.909995 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea7d09-7a91-4ba6-a1c3-0feb18982080 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:52.910484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea7d09-7a91-4ba6-a1c3-0feb18982080 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:52.910954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea7d09-7a91-4ba6-a1c3-0feb18982080 None admin] PATH_INFO: `/v3/groups/b527ebb2c54c4da7ae589da3c6283f65` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:52.917877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea7d09-7a91-4ba6-a1c3-0feb18982080 None admin] RBAC: Authorizing `identity:get_group(group_id=b527ebb2c54c4da7ae589da3c6283f65)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:52.924352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea7d09-7a91-4ba6-a1c3-0feb18982080 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:52.924352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 185/378] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:03:52 2022] GET /identity/v3/groups/b527ebb2c54c4da7ae589da3c6283f65 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:52.933806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea7d09-7a91-4ba6-a1c3-0feb18982080 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:52.958467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:52.980149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b85a9-4cc8-4901-aee8-b0e0494e9c0a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:53.054897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea7d09-7a91-4ba6-a1c3-0feb18982080 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:53.092787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a80e32-7ffe-4914-bc98-77a47871ad8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:53.093762 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a80e32-7ffe-4914-bc98-77a47871ad8f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:53.094187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a80e32-7ffe-4914-bc98-77a47871ad8f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:53.094696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a80e32-7ffe-4914-bc98-77a47871ad8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:53.096230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a80e32-7ffe-4914-bc98-77a47871ad8f None admin] RBAC: Adding query filter params (group.id=b527ebb2c54c4da7ae589da3c6283f65, role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=4f97db8bc2c2455d85720c36a1ed180d) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:53.096715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a80e32-7ffe-4914-bc98-77a47871ad8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:53.098373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a80e32-7ffe-4914-bc98-77a47871ad8f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:53.104725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 186/379] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 7 16:03:52 2022] GET /identity/v3/role_assignments?group.id=b527ebb2c54c4da7ae589da3c6283f65&scope.project.id=4f97db8bc2c2455d85720c36a1ed180d&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:53.112683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373d66d-c444-4430-bb1c-67c5b6258560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:53.114245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373d66d-c444-4430-bb1c-67c5b6258560 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:53.114245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373d66d-c444-4430-bb1c-67c5b6258560 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:53.114245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373d66d-c444-4430-bb1c-67c5b6258560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:53.115206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373d66d-c444-4430-bb1c-67c5b6258560 None admin] RBAC: Adding query filter params (group.id=c3893fedc58b4b7fa399c12d2c3afcb6, role.id=f8c4be33fa2f4e69a3de71a3c4d7b179, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:53.115206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373d66d-c444-4430-bb1c-67c5b6258560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:53.116589 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373d66d-c444-4430-bb1c-67c5b6258560 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:53.129957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 194/380] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 7 16:03:52 2022] GET /identity/v3/role_assignments?group.id=c3893fedc58b4b7fa399c12d2c3afcb6&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 260 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:53.357750 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc71ff6-2692-4ca4-b993-71af18b2c260 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:53.358667 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc71ff6-2692-4ca4-b993-71af18b2c260 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:53.359126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc71ff6-2692-4ca4-b993-71af18b2c260 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:53.360790 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 187/381] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:53.366763 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9873f2-4277-4cf1-a313-6eee6f599a94 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:53.367670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9873f2-4277-4cf1-a313-6eee6f599a94 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:53.368179 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9873f2-4277-4cf1-a313-6eee6f599a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:53.393759 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-9e9873f2-4277-4cf1-a313-6eee6f599a94 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:53.395406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9e9873f2-4277-4cf1-a313-6eee6f599a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:53.476658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 195/382] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:53.987354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc71ff6-2692-4ca4-b993-71af18b2c260 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:54.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:54.091031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529da3c-44ef-4a01-a579-c7f87df80f1a None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:54.091690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529da3c-44ef-4a01-a579-c7f87df80f1a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:54.092233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529da3c-44ef-4a01-a579-c7f87df80f1a None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:54.094139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 196/383] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:54.099134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffc17c-be3f-4650-a4e0-e636296f9d8b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:54.099367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffc17c-be3f-4650-a4e0-e636296f9d8b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:54.099865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffc17c-be3f-4650-a4e0-e636296f9d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:54.129360 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-87ffc17c-be3f-4650-a4e0-e636296f9d8b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:54.130619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-87ffc17c-be3f-4650-a4e0-e636296f9d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:54.133365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc71ff6-2692-4ca4-b993-71af18b2c260 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:54.211545 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418218b-589d-401f-8fc1-8f37d5a406bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:54.212219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418218b-589d-401f-8fc1-8f37d5a406bc None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:54.212497 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418218b-589d-401f-8fc1-8f37d5a406bc None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:54.212769 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418218b-589d-401f-8fc1-8f37d5a406bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:54.237296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418218b-589d-401f-8fc1-8f37d5a406bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-7418218b-589d-401f-8fc1-8f37d5a406bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:54.243156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:54.244110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:54.244952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:54.247476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 188/385] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:54.252760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 197/385] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:54.254407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418218b-589d-401f-8fc1-8f37d5a406bc None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:54.412353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418218b-589d-401f-8fc1-8f37d5a406bc None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:54.427960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:54.456529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78ceb5-8e62-422e-9671-74b7215d0b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:54.457134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78ceb5-8e62-422e-9671-74b7215d0b00 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:54.457414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78ceb5-8e62-422e-9671-74b7215d0b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:54.457683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78ceb5-8e62-422e-9671-74b7215d0b00 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:54.458332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78ceb5-8e62-422e-9671-74b7215d0b00 None admin] RBAC: Adding query filter params (name=member) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:54.458631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78ceb5-8e62-422e-9671-74b7215d0b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:54.460201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78ceb5-8e62-422e-9671-74b7215d0b00 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:54.473841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 189/386] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:54.479365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffc17c-be3f-4650-a4e0-e636296f9d8b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:54.505343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:54.519338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78ceb5-8e62-422e-9671-74b7215d0b00 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:54.639673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffc17c-be3f-4650-a4e0-e636296f9d8b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:54.682152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78ceb5-8e62-422e-9671-74b7215d0b00 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:54.686447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5592a4-15b9-4eee-930e-657e74613ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:54.687422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5592a4-15b9-4eee-930e-657e74613ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:54.687943 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5592a4-15b9-4eee-930e-657e74613ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:54.688429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5592a4-15b9-4eee-930e-657e74613ad7 None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:54.691182 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5592a4-15b9-4eee-930e-657e74613ad7 None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:54.692837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5592a4-15b9-4eee-930e-657e74613ad7 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:54.695503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 198/387] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:03:54 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:54.707283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5592a4-15b9-4eee-930e-657e74613ad7 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:54.760803 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb11b0-dc4f-4b72-9a7b-54b1a724477f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:54.763350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb11b0-dc4f-4b72-9a7b-54b1a724477f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:54.763911 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb11b0-dc4f-4b72-9a7b-54b1a724477f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:54.764463 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb11b0-dc4f-4b72-9a7b-54b1a724477f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:54.784720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb11b0-dc4f-4b72-9a7b-54b1a724477f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-e4eb11b0-dc4f-4b72-9a7b-54b1a724477f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:54.791578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:54.792476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:54.793272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:54.795935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 190/388] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:03:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:03:54.810331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb11b0-dc4f-4b72-9a7b-54b1a724477f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:54.847444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:54.858417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5592a4-15b9-4eee-930e-657e74613ad7 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:54.865076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:54.894152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e71b0c-91c2-4280-9a59-3ef87106fda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:54.895906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e71b0c-91c2-4280-9a59-3ef87106fda2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:54.895906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e71b0c-91c2-4280-9a59-3ef87106fda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:54.895906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e71b0c-91c2-4280-9a59-3ef87106fda2 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:54.896513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e71b0c-91c2-4280-9a59-3ef87106fda2 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:54.898118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e71b0c-91c2-4280-9a59-3ef87106fda2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:54.899914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 199/389] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:54 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:54.905391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e71b0c-91c2-4280-9a59-3ef87106fda2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:54.989036 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb11b0-dc4f-4b72-9a7b-54b1a724477f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:55.022920 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e71b0c-91c2-4280-9a59-3ef87106fda2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:55.059483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbaccb-d32b-4bbc-b8d6-5028aa735751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:55.060184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbaccb-d32b-4bbc-b8d6-5028aa735751 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:55.060573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbaccb-d32b-4bbc-b8d6-5028aa735751 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:55.060858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbaccb-d32b-4bbc-b8d6-5028aa735751 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0/groups/c3893fedc58b4b7fa399c12d2c3afcb6/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:55.062821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbaccb-d32b-4bbc-b8d6-5028aa735751 None admin] RBAC: Authorizing `identity:create_grant(project_id=53b19d145ddf4f3c950098b6269b33e0, group_id=c3893fedc58b4b7fa399c12d2c3afcb6, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:55.064329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbaccb-d32b-4bbc-b8d6-5028aa735751 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:55.078997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf3ffb-f2bc-40ba-aadf-c690a76e477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:55.084726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf3ffb-f2bc-40ba-aadf-c690a76e477c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:55.084726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf3ffb-f2bc-40ba-aadf-c690a76e477c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:55.084726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cf3ffb-f2bc-40ba-aadf-c690a76e477c None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:55.084726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf3ffb-f2bc-40ba-aadf-c690a76e477c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:55.084726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf3ffb-f2bc-40ba-aadf-c690a76e477c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:55.084726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cf3ffb-f2bc-40ba-aadf-c690a76e477c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:55.092382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 191/390] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:03:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:55.098477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf3ffb-f2bc-40ba-aadf-c690a76e477c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:55.138212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 200/391] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Sep 7 16:03:54 2022] PUT /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0/groups/c3893fedc58b4b7fa399c12d2c3afcb6/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:55.223613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:55.230261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cf3ffb-f2bc-40ba-aadf-c690a76e477c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:55.280289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c038d-ebe7-4eef-a1ce-dcc573f26242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:55.280856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c038d-ebe7-4eef-a1ce-dcc573f26242 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:55.281136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c038d-ebe7-4eef-a1ce-dcc573f26242 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:55.281449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c038d-ebe7-4eef-a1ce-dcc573f26242 None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:55.287111 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c038d-ebe7-4eef-a1ce-dcc573f26242 None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:55.287111 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c038d-ebe7-4eef-a1ce-dcc573f26242 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:55.289346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 192/392] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:03:55 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:55.296167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbaccb-d32b-4bbc-b8d6-5028aa735751 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:55.324474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:55.435701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbaccb-d32b-4bbc-b8d6-5028aa735751 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:55.472136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876758c-c6f0-4d47-b903-65c089f3a0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:55.472832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876758c-c6f0-4d47-b903-65c089f3a0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:55.473116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876758c-c6f0-4d47-b903-65c089f3a0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:55.473389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876758c-c6f0-4d47-b903-65c089f3a0c0 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:55.474527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876758c-c6f0-4d47-b903-65c089f3a0c0 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:55.476020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876758c-c6f0-4d47-b903-65c089f3a0c0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:55.478080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 201/393] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:55 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:55.483842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c038d-ebe7-4eef-a1ce-dcc573f26242 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:55.494366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31dbb2-e85e-4947-89f4-0e08b492eb2d None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:55.494891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31dbb2-e85e-4947-89f4-0e08b492eb2d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:55.494891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31dbb2-e85e-4947-89f4-0e08b492eb2d None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:55.496556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 202/394] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:55.502261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829eae99-6b93-4233-8a7f-fa6d58f4a539 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:55.502552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829eae99-6b93-4233-8a7f-fa6d58f4a539 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:55.502830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829eae99-6b93-4233-8a7f-fa6d58f4a539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:55.524691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-829eae99-6b93-4233-8a7f-fa6d58f4a539 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:55.525900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-829eae99-6b93-4233-8a7f-fa6d58f4a539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:55.623723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 203/395] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:55.657334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c038d-ebe7-4eef-a1ce-dcc573f26242 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:55.681834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416cdbd-6537-4afa-8f7b-71693b5876d0 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:55.682164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416cdbd-6537-4afa-8f7b-71693b5876d0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:55.682386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416cdbd-6537-4afa-8f7b-71693b5876d0 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:55.683879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 204/396] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:55.688934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecbd83-d6be-4549-a035-314c7a8c3f4a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:55.689846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecbd83-d6be-4549-a035-314c7a8c3f4a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:55.690128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecbd83-d6be-4549-a035-314c7a8c3f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:55.713924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1cecbd83-d6be-4549-a035-314c7a8c3f4a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:55.715389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1cecbd83-d6be-4549-a035-314c7a8c3f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:55.732746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:55.735930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17932fc5-7f1c-43b9-9162-16b090260128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:55.737187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17932fc5-7f1c-43b9-9162-16b090260128 None admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:55.737469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17932fc5-7f1c-43b9-9162-16b090260128 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:55.737776 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17932fc5-7f1c-43b9-9162-16b090260128 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0/groups/c3893fedc58b4b7fa399c12d2c3afcb6/roles/f8c4be33fa2f4e69a3de71a3c4d7b179` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:55.741051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17932fc5-7f1c-43b9-9162-16b090260128 None admin] RBAC: Authorizing `identity:create_grant(project_id=53b19d145ddf4f3c950098b6269b33e0, group_id=c3893fedc58b4b7fa399c12d2c3afcb6, role_id=f8c4be33fa2f4e69a3de71a3c4d7b179)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:55.742555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17932fc5-7f1c-43b9-9162-16b090260128 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:55.802755 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 205/397] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:55.844051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 193/398] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Sep 7 16:03:55 2022] PUT /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0/groups/c3893fedc58b4b7fa399c12d2c3afcb6/roles/f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:55.985714 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecbd83-d6be-4549-a035-314c7a8c3f4a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:56.089178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17932fc5-7f1c-43b9-9162-16b090260128 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:56.101867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecbd83-d6be-4549-a035-314c7a8c3f4a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:56.140137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:56.144789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d02fe9-43f5-4e5d-9a50-d378800351a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:56.145772 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d02fe9-43f5-4e5d-9a50-d378800351a7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:56.146050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d02fe9-43f5-4e5d-9a50-d378800351a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:56.146812 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d02fe9-43f5-4e5d-9a50-d378800351a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:56.162323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d02fe9-43f5-4e5d-9a50-d378800351a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-b7d02fe9-43f5-4e5d-9a50-d378800351a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:56.169082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:56.170074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:03:56.170872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 206/399] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:56.177422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d02fe9-43f5-4e5d-9a50-d378800351a7 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:56.182690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:56.214389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17932fc5-7f1c-43b9-9162-16b090260128 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:56.253852 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3e432-0b27-4b25-a964-011579863007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:56.254579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3e432-0b27-4b25-a964-011579863007 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:56.254959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3e432-0b27-4b25-a964-011579863007 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:56.255290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3e432-0b27-4b25-a964-011579863007 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:56.277775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3e432-0b27-4b25-a964-011579863007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-5ed3e432-0b27-4b25-a964-011579863007 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:56.283584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:56.285276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 194/400] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:03:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:03:56.286681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4132701-3695-465e-a306-74644b5bccf8 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:56.287430 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4132701-3695-465e-a306-74644b5bccf8 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:56.287430 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4132701-3695-465e-a306-74644b5bccf8 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:56.293517 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 195/401] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:56.293935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d02fe9-43f5-4e5d-9a50-d378800351a7 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:56.294784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4132701-3695-465e-a306-74644b5bccf8 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:56.334003 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca14193-aa99-4497-94a4-b6825f6e1927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:56.334616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca14193-aa99-4497-94a4-b6825f6e1927 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:56.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca14193-aa99-4497-94a4-b6825f6e1927 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:56.335162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca14193-aa99-4497-94a4-b6825f6e1927 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:56.335847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca14193-aa99-4497-94a4-b6825f6e1927 None admin] RBAC: Adding query filter params (name=member) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:56.336146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca14193-aa99-4497-94a4-b6825f6e1927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:56.337515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca14193-aa99-4497-94a4-b6825f6e1927 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:56.346935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 207/402] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:56.352406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd3972-82d4-4489-b20b-a0c5b50616ac None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:56.352816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd3972-82d4-4489-b20b-a0c5b50616ac None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:56.352972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd3972-82d4-4489-b20b-a0c5b50616ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:56.372276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-a8cd3972-82d4-4489-b20b-a0c5b50616ac None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:56.373463 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a8cd3972-82d4-4489-b20b-a0c5b50616ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:56.396176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:56.416445 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4132701-3695-465e-a306-74644b5bccf8 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:56.442823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:56.450640 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 208/403] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:56.451703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd3972-82d4-4489-b20b-a0c5b50616ac None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:56.465305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449a1a0-713c-4514-9dc5-c550b86d2247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:56.466181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449a1a0-713c-4514-9dc5-c550b86d2247 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:56.466685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449a1a0-713c-4514-9dc5-c550b86d2247 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:56.467210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f449a1a0-713c-4514-9dc5-c550b86d2247 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:56.468091 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449a1a0-713c-4514-9dc5-c550b86d2247 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:56.468582 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449a1a0-713c-4514-9dc5-c550b86d2247 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:56.471400 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f449a1a0-713c-4514-9dc5-c550b86d2247 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:56.489528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 196/404] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:03:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:56.495487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449a1a0-713c-4514-9dc5-c550b86d2247 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:56.592996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd3972-82d4-4489-b20b-a0c5b50616ac None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:56.629244 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26464dd3-b29e-49be-a124-3c1008b5ba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:56.630245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26464dd3-b29e-49be-a124-3c1008b5ba0a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:56.630729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26464dd3-b29e-49be-a124-3c1008b5ba0a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:56.631190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26464dd3-b29e-49be-a124-3c1008b5ba0a None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:56.632619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26464dd3-b29e-49be-a124-3c1008b5ba0a None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:56.634238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26464dd3-b29e-49be-a124-3c1008b5ba0a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:56.636452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 209/405] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:03:56 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:56.641980 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26464dd3-b29e-49be-a124-3c1008b5ba0a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:56.713753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:56.721992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f449a1a0-713c-4514-9dc5-c550b86d2247 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:56.756133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26464dd3-b29e-49be-a124-3c1008b5ba0a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:56.796561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ee8e2-564e-49db-9395-0f8dbf6244eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:56.797576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee8e2-564e-49db-9395-0f8dbf6244eb None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:56.798060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee8e2-564e-49db-9395-0f8dbf6244eb None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:56.798564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ee8e2-564e-49db-9395-0f8dbf6244eb None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:56.799936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ee8e2-564e-49db-9395-0f8dbf6244eb None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:56.801329 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb79e2f-b256-4257-ac2c-0e265f1f8f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:56.801805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ee8e2-564e-49db-9395-0f8dbf6244eb None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:56.801972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb79e2f-b256-4257-ac2c-0e265f1f8f6c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:56.802239 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb79e2f-b256-4257-ac2c-0e265f1f8f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:56.802514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb79e2f-b256-4257-ac2c-0e265f1f8f6c None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:56.803637 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb79e2f-b256-4257-ac2c-0e265f1f8f6c None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:56.804818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 210/406] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:56 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:56.809786 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb79e2f-b256-4257-ac2c-0e265f1f8f6c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:56.810844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ee8e2-564e-49db-9395-0f8dbf6244eb None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:56.811845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 197/407] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:03:56 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:56.813598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb79e2f-b256-4257-ac2c-0e265f1f8f6c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:56.837822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:56.942409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb79e2f-b256-4257-ac2c-0e265f1f8f6c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:56.946711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ee8e2-564e-49db-9395-0f8dbf6244eb None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:56.986597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:56.989345 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d0aad-6fa6-464a-b0d5-3addc467e9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:56.990306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d0aad-6fa6-464a-b0d5-3addc467e9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:56.990803 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d0aad-6fa6-464a-b0d5-3addc467e9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:56.991273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d0aad-6fa6-464a-b0d5-3addc467e9d1 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:56.991983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f4b5e-dbb2-49f6-a650-eec49c084a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:56.992767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f4b5e-dbb2-49f6-a650-eec49c084a53 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:56.993046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f4b5e-dbb2-49f6-a650-eec49c084a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:56.993323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f4b5e-dbb2-49f6-a650-eec49c084a53 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/groups/c3893fedc58b4b7fa399c12d2c3afcb6/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:56.993663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d0aad-6fa6-464a-b0d5-3addc467e9d1 None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:56.995328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f4b5e-dbb2-49f6-a650-eec49c084a53 None admin] RBAC: Authorizing `identity:create_grant(project_id=225b5e655135449ab5cb4e25c5c9fca4, group_id=c3893fedc58b4b7fa399c12d2c3afcb6, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:56.995708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d0aad-6fa6-464a-b0d5-3addc467e9d1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:56.996813 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f4b5e-dbb2-49f6-a650-eec49c084a53 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:56.997921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 198/408] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:56 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:56.998884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d0aad-6fa6-464a-b0d5-3addc467e9d1 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:57.075997 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 211/409] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Sep 7 16:03:56 2022] PUT /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/groups/c3893fedc58b4b7fa399c12d2c3afcb6/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:57.077509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f4b5e-dbb2-49f6-a650-eec49c084a53 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:57.098808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:57.127687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d0aad-6fa6-464a-b0d5-3addc467e9d1 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:57.169454 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f29ef1-72d6-437b-bc40-cec1050f8b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:57.169873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f29ef1-72d6-437b-bc40-cec1050f8b6f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:57.170149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f29ef1-72d6-437b-bc40-cec1050f8b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:57.170420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f29ef1-72d6-437b-bc40-cec1050f8b6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:57.195627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f29ef1-72d6-437b-bc40-cec1050f8b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-f5f29ef1-72d6-437b-bc40-cec1050f8b6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:57.202014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:57.202979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:57.204468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 199/410] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:03:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:57.208971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f29ef1-72d6-437b-bc40-cec1050f8b6f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:57.305863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f4b5e-dbb2-49f6-a650-eec49c084a53 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:57.325517 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f29ef1-72d6-437b-bc40-cec1050f8b6f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:57.363313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcc4e7-e6f4-4b48-963c-fc11c8419917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:57.364181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcc4e7-e6f4-4b48-963c-fc11c8419917 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:57.364461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcc4e7-e6f4-4b48-963c-fc11c8419917 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:57.364730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcc4e7-e6f4-4b48-963c-fc11c8419917 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/groups/c3893fedc58b4b7fa399c12d2c3afcb6/roles/f8c4be33fa2f4e69a3de71a3c4d7b179` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:57.367368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcc4e7-e6f4-4b48-963c-fc11c8419917 None admin] RBAC: Authorizing `identity:create_grant(project_id=225b5e655135449ab5cb4e25c5c9fca4, group_id=c3893fedc58b4b7fa399c12d2c3afcb6, role_id=f8c4be33fa2f4e69a3de71a3c4d7b179)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:57.368837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcc4e7-e6f4-4b48-963c-fc11c8419917 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:57.370153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7140da-36a6-4c08-881c-d46f26a5f340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:57.370772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7140da-36a6-4c08-881c-d46f26a5f340 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:57.371330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7140da-36a6-4c08-881c-d46f26a5f340 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:57.371610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7140da-36a6-4c08-881c-d46f26a5f340 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:57.373038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7140da-36a6-4c08-881c-d46f26a5f340 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:57.373356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7140da-36a6-4c08-881c-d46f26a5f340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:57.374831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7140da-36a6-4c08-881c-d46f26a5f340 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:57.386131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 200/411] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:03:57.391981 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7140da-36a6-4c08-881c-d46f26a5f340 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:57.395036 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:57.449488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 212/412] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Sep 7 16:03:57 2022] PUT /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4/groups/c3893fedc58b4b7fa399c12d2c3afcb6/roles/f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:03:57.511044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7140da-36a6-4c08-881c-d46f26a5f340 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:57.549438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e70b3d-1625-4c61-971a-da3a27b37671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:57.550415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e70b3d-1625-4c61-971a-da3a27b37671 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:57.550865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e70b3d-1625-4c61-971a-da3a27b37671 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:57.551308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e70b3d-1625-4c61-971a-da3a27b37671 None admin] PATH_INFO: `/v3/groups/b527ebb2c54c4da7ae589da3c6283f65` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:57.552668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e70b3d-1625-4c61-971a-da3a27b37671 None admin] RBAC: Authorizing `identity:get_group(group_id=b527ebb2c54c4da7ae589da3c6283f65)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:57.554372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e70b3d-1625-4c61-971a-da3a27b37671 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:57.556950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 201/413] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:03:57 2022] GET /identity/v3/groups/b527ebb2c54c4da7ae589da3c6283f65 => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:57.562933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcc4e7-e6f4-4b48-963c-fc11c8419917 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:57.634724 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:57.689173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d74e-aa19-468f-90b7-e8913079122a None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:57.689443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d74e-aa19-468f-90b7-e8913079122a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:57.689716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553d74e-aa19-468f-90b7-e8913079122a None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:57.691478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 202/414] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:03:57.697086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df06c-af21-497e-b0e8-6ee68813a5a2 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:57.697360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df06c-af21-497e-b0e8-6ee68813a5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:57.697634 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df06c-af21-497e-b0e8-6ee68813a5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:57.715373 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcc4e7-e6f4-4b48-963c-fc11c8419917 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:57.724663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-cb5df06c-af21-497e-b0e8-6ee68813a5a2 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:57.725984 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-cb5df06c-af21-497e-b0e8-6ee68813a5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:57.754846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7cfa7-2269-40b0-8593-8cbaabad95a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:57.756097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7cfa7-2269-40b0-8593-8cbaabad95a2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:57.756570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7cfa7-2269-40b0-8593-8cbaabad95a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:57.757029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7cfa7-2269-40b0-8593-8cbaabad95a2 None admin] PATH_INFO: `/v3/projects/4f97db8bc2c2455d85720c36a1ed180d` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:57.759909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7cfa7-2269-40b0-8593-8cbaabad95a2 None admin] RBAC: Authorizing `identity:get_project(project_id=4f97db8bc2c2455d85720c36a1ed180d)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:57.761555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7cfa7-2269-40b0-8593-8cbaabad95a2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:57.763676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 213/415] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:57 2022] GET /identity/v3/projects/4f97db8bc2c2455d85720c36a1ed180d => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:57.769500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7cfa7-2269-40b0-8593-8cbaabad95a2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:57.810995 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:57.829103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 203/416] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 07 16:03:57.905681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7cfa7-2269-40b0-8593-8cbaabad95a2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:57.932884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:57.949777 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568830-5f56-4a6a-b1a4-d7647af4c7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:57.963443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568830-5f56-4a6a-b1a4-d7647af4c7e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:57.963443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568830-5f56-4a6a-b1a4-d7647af4c7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:57.963443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568830-5f56-4a6a-b1a4-d7647af4c7e0 None admin] PATH_INFO: `/v3/projects/4f97db8bc2c2455d85720c36a1ed180d/groups/b527ebb2c54c4da7ae589da3c6283f65/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:57.963443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57568830-5f56-4a6a-b1a4-d7647af4c7e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4f97db8bc2c2455d85720c36a1ed180d, group_id=b527ebb2c54c4da7ae589da3c6283f65, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:57.963443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57568830-5f56-4a6a-b1a4-d7647af4c7e0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:58.084236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 214/417] 10.222.0.15 () {58 vars in 1442 bytes} [Wed Sep 7 16:03:57 2022] PUT /identity/v3/projects/4f97db8bc2c2455d85720c36a1ed180d/groups/b527ebb2c54c4da7ae589da3c6283f65/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:03:58.141199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5df06c-af21-497e-b0e8-6ee68813a5a2 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:58.294598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5df06c-af21-497e-b0e8-6ee68813a5a2 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:58.337428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed63261-1420-478e-87a3-8d8b69aa8011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:58.338066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed63261-1420-478e-87a3-8d8b69aa8011 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:58.338348 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed63261-1420-478e-87a3-8d8b69aa8011 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:58.338622 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed63261-1420-478e-87a3-8d8b69aa8011 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:58.349210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed63261-1420-478e-87a3-8d8b69aa8011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-2ed63261-1420-478e-87a3-8d8b69aa8011 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:58.354073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:58.354953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:58.355745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 204/418] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:03:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:03:58.361365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568830-5f56-4a6a-b1a4-d7647af4c7e0 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:58.494594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568830-5f56-4a6a-b1a4-d7647af4c7e0 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:58.533641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2efa01-fe39-49ae-967f-f11daf0fd973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:58.534269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2efa01-fe39-49ae-967f-f11daf0fd973 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:58.534552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2efa01-fe39-49ae-967f-f11daf0fd973 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:58.534832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2efa01-fe39-49ae-967f-f11daf0fd973 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:58.535498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2efa01-fe39-49ae-967f-f11daf0fd973 None admin] RBAC: Adding query filter params (name=member) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:58.535821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2efa01-fe39-49ae-967f-f11daf0fd973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:58.537217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2efa01-fe39-49ae-967f-f11daf0fd973 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:58.552120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 215/419] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:03:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:58.557986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed63261-1420-478e-87a3-8d8b69aa8011 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:58.603591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008defb-4b3f-44ee-b79c-48ea2589433d None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:58.604014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008defb-4b3f-44ee-b79c-48ea2589433d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:58.604543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008defb-4b3f-44ee-b79c-48ea2589433d None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:58.606136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 216/420] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:58.612428 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec9dab-8f7d-46fb-a5a1-cd5133420372 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:58.612715 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec9dab-8f7d-46fb-a5a1-cd5133420372 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:58.612995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec9dab-8f7d-46fb-a5a1-cd5133420372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:58.636131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-a9ec9dab-8f7d-46fb-a5a1-cd5133420372 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:58.637895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a9ec9dab-8f7d-46fb-a5a1-cd5133420372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:58.691881 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:58.695400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:58.744772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed63261-1420-478e-87a3-8d8b69aa8011 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:58.777023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 217/421] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:03:58.797158 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fb11a-32e2-4025-b21e-a39a21ac979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:58.797781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fb11a-32e2-4025-b21e-a39a21ac979d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:58.798064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fb11a-32e2-4025-b21e-a39a21ac979d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:58.798910 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9fb11a-32e2-4025-b21e-a39a21ac979d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:58.878341 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fb11a-32e2-4025-b21e-a39a21ac979d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:58.879823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9fb11a-32e2-4025-b21e-a39a21ac979d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:58.948142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:58.981808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 205/422] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:03:58.988036 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec9dab-8f7d-46fb-a5a1-cd5133420372 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:59.062196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fb11a-32e2-4025-b21e-a39a21ac979d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:59.103502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec9dab-8f7d-46fb-a5a1-cd5133420372 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:59.141298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacf85e-d983-4a40-8e8a-4fb9790957e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:59.142062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacf85e-d983-4a40-8e8a-4fb9790957e6 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:59.142515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacf85e-d983-4a40-8e8a-4fb9790957e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:59.142885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bacf85e-d983-4a40-8e8a-4fb9790957e6 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:59.148953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bacf85e-d983-4a40-8e8a-4fb9790957e6 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:59.152572 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bacf85e-d983-4a40-8e8a-4fb9790957e6 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:59.163707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 218/423] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:03:58 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:59.164622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacf85e-d983-4a40-8e8a-4fb9790957e6 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:59.221501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:59.269071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9fb11a-32e2-4025-b21e-a39a21ac979d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:59.287729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bacf85e-d983-4a40-8e8a-4fb9790957e6 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:59.326823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bdc81-c256-4bbe-bf17-8b5d6f63b1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:59.327460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdc81-c256-4bbe-bf17-8b5d6f63b1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:59.327740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdc81-c256-4bbe-bf17-8b5d6f63b1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:59.328046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bdc81-c256-4bbe-bf17-8b5d6f63b1c0 None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:59.329443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bdc81-c256-4bbe-bf17-8b5d6f63b1c0 None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:59.332181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481bdc81-c256-4bbe-bf17-8b5d6f63b1c0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:59.333720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 219/424] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:03:59 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:59.339883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bdc81-c256-4bbe-bf17-8b5d6f63b1c0 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:59.349974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48079c-1451-4515-9f98-c01a6b8a78af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:59.350625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48079c-1451-4515-9f98-c01a6b8a78af None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:59.350899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48079c-1451-4515-9f98-c01a6b8a78af None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:59.351168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48079c-1451-4515-9f98-c01a6b8a78af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:59.377949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48079c-1451-4515-9f98-c01a6b8a78af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-9e48079c-1451-4515-9f98-c01a6b8a78af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:03:59.382851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:03:59.383726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:03:59.384544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 206/425] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:03:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:03:59.391409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48079c-1451-4515-9f98-c01a6b8a78af None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:59.433768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:59.452194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481bdc81-c256-4bbe-bf17-8b5d6f63b1c0 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:59.463128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:59.494323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220435b-ad29-4d17-9840-8272e46795e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:59.494955 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220435b-ad29-4d17-9840-8272e46795e8 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:59.496018 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220435b-ad29-4d17-9840-8272e46795e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:59.496306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220435b-ad29-4d17-9840-8272e46795e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:59.497258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220435b-ad29-4d17-9840-8272e46795e8 None admin] RBAC: Adding query filter params (group.id=c3893fedc58b4b7fa399c12d2c3afcb6, role.id=f463de99ba174eab9396eef514e34e15, scope.project.id=53b19d145ddf4f3c950098b6269b33e0) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:59.497576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220435b-ad29-4d17-9840-8272e46795e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:59.499128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220435b-ad29-4d17-9840-8272e46795e8 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:59.507458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 220/426] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 7 16:03:59 2022] GET /identity/v3/role_assignments?group.id=c3893fedc58b4b7fa399c12d2c3afcb6&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=f463de99ba174eab9396eef514e34e15 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:59.609402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48079c-1451-4515-9f98-c01a6b8a78af None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:59.629137 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c422a-e4b4-4244-86fa-09390996620a None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:59.629730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c422a-e4b4-4244-86fa-09390996620a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:59.629837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c422a-e4b4-4244-86fa-09390996620a None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:59.631298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 221/427] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:03:59.638467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495060b6-4c00-4153-bc36-bc38b85323eb None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:59.639526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495060b6-4c00-4153-bc36-bc38b85323eb None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:59.640048 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495060b6-4c00-4153-bc36-bc38b85323eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:59.667176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-495060b6-4c00-4153-bc36-bc38b85323eb None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:59.668347 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-495060b6-4c00-4153-bc36-bc38b85323eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:59.674486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274701bc-61b0-45f3-b958-31f83cc18090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:59.675108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274701bc-61b0-45f3-b958-31f83cc18090 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:59.675384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274701bc-61b0-45f3-b958-31f83cc18090 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:59.675655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274701bc-61b0-45f3-b958-31f83cc18090 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:59.676366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274701bc-61b0-45f3-b958-31f83cc18090 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:03:59.676666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274701bc-61b0-45f3-b958-31f83cc18090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:59.678077 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274701bc-61b0-45f3-b958-31f83cc18090 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:59.696563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 207/428] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:03:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:03:59.702424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274701bc-61b0-45f3-b958-31f83cc18090 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:59.778378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 222/429] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:03:59.805333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:59.824838 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274701bc-61b0-45f3-b958-31f83cc18090 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:59.860672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b20a3-db9b-42cc-a0b0-1c261c71605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:59.861348 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b20a3-db9b-42cc-a0b0-1c261c71605d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:59.861603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b20a3-db9b-42cc-a0b0-1c261c71605d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:59.861879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b20a3-db9b-42cc-a0b0-1c261c71605d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:59.935537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b20a3-db9b-42cc-a0b0-1c261c71605d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:59.936934 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b20a3-db9b-42cc-a0b0-1c261c71605d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:00.027510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:00.049032 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 208/430] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:03:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 16:04:00.054803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495060b6-4c00-4153-bc36-bc38b85323eb None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:00.105278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b20a3-db9b-42cc-a0b0-1c261c71605d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:00.169172 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495060b6-4c00-4153-bc36-bc38b85323eb None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:00.170899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:00.212249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09b4c2-10e8-4ab8-9d34-7c6103114b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:00.212829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09b4c2-10e8-4ab8-9d34-7c6103114b04 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:00.213112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09b4c2-10e8-4ab8-9d34-7c6103114b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:00.213391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d09b4c2-10e8-4ab8-9d34-7c6103114b04 None admin] PATH_INFO: `/v3/projects/53b19d145ddf4f3c950098b6269b33e0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:00.214591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09b4c2-10e8-4ab8-9d34-7c6103114b04 None admin] RBAC: Authorizing `identity:get_project(project_id=53b19d145ddf4f3c950098b6269b33e0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:00.216106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d09b4c2-10e8-4ab8-9d34-7c6103114b04 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:00.218119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 223/431] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:04:00 2022] GET /identity/v3/projects/53b19d145ddf4f3c950098b6269b33e0 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:00.223305 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09b4c2-10e8-4ab8-9d34-7c6103114b04 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:00.230031 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b20a3-db9b-42cc-a0b0-1c261c71605d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:00.269510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61489619-1084-44de-b3fa-61e0c9b96b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:00.270495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61489619-1084-44de-b3fa-61e0c9b96b0e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:00.271835 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61489619-1084-44de-b3fa-61e0c9b96b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:00.272362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61489619-1084-44de-b3fa-61e0c9b96b0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:00.284520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61489619-1084-44de-b3fa-61e0c9b96b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-61489619-1084-44de-b3fa-61e0c9b96b0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:00.289861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:00.290757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 16:04:00.291540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:00.297347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 209/432] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:00.300876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61489619-1084-44de-b3fa-61e0c9b96b0e None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:00.351505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:00.358510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d09b4c2-10e8-4ab8-9d34-7c6103114b04 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:00.401507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de772234-86ed-439b-a2c7-0ece141d8497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:00.402233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de772234-86ed-439b-a2c7-0ece141d8497 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:00.402583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de772234-86ed-439b-a2c7-0ece141d8497 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:00.402922 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de772234-86ed-439b-a2c7-0ece141d8497 None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:00.404162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de772234-86ed-439b-a2c7-0ece141d8497 None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:00.405609 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de772234-86ed-439b-a2c7-0ece141d8497 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:00.407496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 224/433] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:04:00 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:00.409674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d06216-4fcd-4947-a690-1725783ca613 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:00.409951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d06216-4fcd-4947-a690-1725783ca613 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:00.410219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d06216-4fcd-4947-a690-1725783ca613 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:00.411680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 225/434] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:00.416406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d06216-4fcd-4947-a690-1725783ca613 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:00.428657 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61489619-1084-44de-b3fa-61e0c9b96b0e None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:00.471979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:00.472516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78185-316c-4729-9207-e3a1fc3ac3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:00.473230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78185-316c-4729-9207-e3a1fc3ac3ea None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:00.473636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78185-316c-4729-9207-e3a1fc3ac3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:00.474326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78185-316c-4729-9207-e3a1fc3ac3ea None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:00.474781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78185-316c-4729-9207-e3a1fc3ac3ea None admin] RBAC: Adding query filter params (name=member) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:00.475178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78185-316c-4729-9207-e3a1fc3ac3ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:00.476716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78185-316c-4729-9207-e3a1fc3ac3ea None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:00.485210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 210/435] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:04:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:00.486595 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adecc53-61af-4911-9d79-db2bf387622f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:00.486950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adecc53-61af-4911-9d79-db2bf387622f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:00.487389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adecc53-61af-4911-9d79-db2bf387622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:00.509244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-6adecc53-61af-4911-9d79-db2bf387622f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:00.510435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6adecc53-61af-4911-9d79-db2bf387622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:00.542021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d06216-4fcd-4947-a690-1725783ca613 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:00.579432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a23b0f-8c44-4565-adcf-4512cbf31002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:00.580611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a23b0f-8c44-4565-adcf-4512cbf31002 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:00.581007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a23b0f-8c44-4565-adcf-4512cbf31002 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:00.581175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a23b0f-8c44-4565-adcf-4512cbf31002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:00.581881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a23b0f-8c44-4565-adcf-4512cbf31002 None admin] RBAC: Adding query filter params (group.id=c3893fedc58b4b7fa399c12d2c3afcb6, role.id=f8c4be33fa2f4e69a3de71a3c4d7b179, scope.project.id=53b19d145ddf4f3c950098b6269b33e0) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:00.582190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a23b0f-8c44-4565-adcf-4512cbf31002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:00.583626 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a23b0f-8c44-4565-adcf-4512cbf31002 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:00.596518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 211/436] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:00.596518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adecc53-61af-4911-9d79-db2bf387622f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:00.600559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 226/437] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 7 16:04:00 2022] GET /identity/v3/role_assignments?group.id=c3893fedc58b4b7fa399c12d2c3afcb6&scope.project.id=53b19d145ddf4f3c950098b6269b33e0&role.id=f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:00.634227 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:00.714596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adecc53-61af-4911-9d79-db2bf387622f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:00.755233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412d2b7-ef71-4391-b037-14ff041978ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:00.756230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d2b7-ef71-4391-b037-14ff041978ea None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:00.756756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d2b7-ef71-4391-b037-14ff041978ea None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:00.757263 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412d2b7-ef71-4391-b037-14ff041978ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:00.832624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412d2b7-ef71-4391-b037-14ff041978ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:00.834345 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412d2b7-ef71-4391-b037-14ff041978ea None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:00.872371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:00.880975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a23b0f-8c44-4565-adcf-4512cbf31002 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:00.963909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 212/438] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:00.965251 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9299cc-c71a-47d4-a813-a50a6f2c2c0c None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:00.965541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9299cc-c71a-47d4-a813-a50a6f2c2c0c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:00.965812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9299cc-c71a-47d4-a813-a50a6f2c2c0c None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:00.967308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 213/439] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:00.971996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9299cc-c71a-47d4-a813-a50a6f2c2c0c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:01.018596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a23b0f-8c44-4565-adcf-4512cbf31002 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:01.020777 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:01.058089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96348fe7-5905-4db0-8820-1544c3714e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:01.058718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96348fe7-5905-4db0-8820-1544c3714e75 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.058998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96348fe7-5905-4db0-8820-1544c3714e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.059270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96348fe7-5905-4db0-8820-1544c3714e75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.069673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96348fe7-5905-4db0-8820-1544c3714e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-96348fe7-5905-4db0-8820-1544c3714e75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:01.074492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:01.075493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:01.076520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 227/440] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:04:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:01.077348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f383d-483b-4888-9890-053d7020bab5 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.077348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f383d-483b-4888-9890-053d7020bab5 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.077525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979f383d-483b-4888-9890-053d7020bab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.084197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9299cc-c71a-47d4-a813-a50a6f2c2c0c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:01.094870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-979f383d-483b-4888-9890-053d7020bab5 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:01.096077 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-979f383d-483b-4888-9890-053d7020bab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:01.118043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6a26e-6168-481a-b5a7-8109797106a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:01.118673 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6a26e-6168-481a-b5a7-8109797106a9 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.118952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6a26e-6168-481a-b5a7-8109797106a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.119226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6a26e-6168-481a-b5a7-8109797106a9 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.120460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6a26e-6168-481a-b5a7-8109797106a9 None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:01.121923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e6a26e-6168-481a-b5a7-8109797106a9 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:01.123534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 214/441] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:04:00 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:01.124322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6a26e-6168-481a-b5a7-8109797106a9 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:01.127363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:01.164565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 228/442] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:01.165393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f383d-483b-4888-9890-053d7020bab5 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:01.238857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6a26e-6168-481a-b5a7-8109797106a9 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:01.258970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:01.270624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979f383d-483b-4888-9890-053d7020bab5 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:01.274808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e28dea-8551-4da6-bdfa-40956648276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:01.275522 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e28dea-8551-4da6-bdfa-40956648276b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.275938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e28dea-8551-4da6-bdfa-40956648276b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.276286 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e28dea-8551-4da6-bdfa-40956648276b None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.277014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e28dea-8551-4da6-bdfa-40956648276b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:01.277380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e28dea-8551-4da6-bdfa-40956648276b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:01.278804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e28dea-8551-4da6-bdfa-40956648276b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:01.287808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 215/443] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:01.293361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e28dea-8551-4da6-bdfa-40956648276b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:01.302180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b52d4-96be-40a9-a042-345fcc1f6eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:01.302813 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b52d4-96be-40a9-a042-345fcc1f6eda None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.303092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b52d4-96be-40a9-a042-345fcc1f6eda None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.303368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b52d4-96be-40a9-a042-345fcc1f6eda None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.304515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b52d4-96be-40a9-a042-345fcc1f6eda None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:01.305938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b52d4-96be-40a9-a042-345fcc1f6eda None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:01.312775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 229/444] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:04:01 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:01.313527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b52d4-96be-40a9-a042-345fcc1f6eda None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:01.366153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:01.404434 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e28dea-8551-4da6-bdfa-40956648276b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:01.418529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b52d4-96be-40a9-a042-345fcc1f6eda None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:01.440660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1825f-f733-49d6-af90-4292d5372e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:01.441273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1825f-f733-49d6-af90-4292d5372e06 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.441557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1825f-f733-49d6-af90-4292d5372e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.441834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1825f-f733-49d6-af90-4292d5372e06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.452813 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c10e3-e81d-4d69-ab8f-9f3f7d652d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:01.453410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c10e3-e81d-4d69-ab8f-9f3f7d652d02 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.453683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c10e3-e81d-4d69-ab8f-9f3f7d652d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.453953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1c10e3-e81d-4d69-ab8f-9f3f7d652d02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.454983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c10e3-e81d-4d69-ab8f-9f3f7d652d02 None admin] RBAC: Adding query filter params (group.id=c3893fedc58b4b7fa399c12d2c3afcb6, role.id=f463de99ba174eab9396eef514e34e15, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:01.455290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c10e3-e81d-4d69-ab8f-9f3f7d652d02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:01.456746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1c10e3-e81d-4d69-ab8f-9f3f7d652d02 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:01.462313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 230/445] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 7 16:04:01 2022] GET /identity/v3/role_assignments?group.id=c3893fedc58b4b7fa399c12d2c3afcb6&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=f463de99ba174eab9396eef514e34e15 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:01.464106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c10e3-e81d-4d69-ab8f-9f3f7d652d02 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:01.493993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:01.518097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1825f-f733-49d6-af90-4292d5372e06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:01.519581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1825f-f733-49d6-af90-4292d5372e06 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:01.583473 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1c10e3-e81d-4d69-ab8f-9f3f7d652d02 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:01.593081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:01.623255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a2095-d302-44a6-a164-67f554f3eff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:01.624314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2095-d302-44a6-a164-67f554f3eff0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.624790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2095-d302-44a6-a164-67f554f3eff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.625254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a2095-d302-44a6-a164-67f554f3eff0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.635269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 216/446] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:01.638290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a2095-d302-44a6-a164-67f554f3eff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:01.640738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1825f-f733-49d6-af90-4292d5372e06 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-f92a2095-d302-44a6-a164-67f554f3eff0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:01.644438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:01.645443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:01.646400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 231/447] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:01.652913 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a2095-d302-44a6-a164-67f554f3eff0 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:01.755328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:01.762610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1825f-f733-49d6-af90-4292d5372e06 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:01.775572 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a2095-d302-44a6-a164-67f554f3eff0 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:01.807697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c827e-c7fc-4701-8870-4b4f10263db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:01.808422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c827e-c7fc-4701-8870-4b4f10263db9 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.808628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c827e-c7fc-4701-8870-4b4f10263db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.808902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c827e-c7fc-4701-8870-4b4f10263db9 None admin] PATH_INFO: `/v3/projects/225b5e655135449ab5cb4e25c5c9fca4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.810019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c827e-c7fc-4701-8870-4b4f10263db9 None admin] RBAC: Authorizing `identity:get_project(project_id=225b5e655135449ab5cb4e25c5c9fca4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:01.811796 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507c827e-c7fc-4701-8870-4b4f10263db9 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:01.813401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 217/448] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:04:01 2022] GET /identity/v3/projects/225b5e655135449ab5cb4e25c5c9fca4 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:01.818511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c827e-c7fc-4701-8870-4b4f10263db9 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:01.821888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0179e-4c5b-426e-8da5-ca4549d134cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:01.822500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0179e-4c5b-426e-8da5-ca4549d134cf None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.822778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0179e-4c5b-426e-8da5-ca4549d134cf None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.823053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0179e-4c5b-426e-8da5-ca4549d134cf None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.823718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0179e-4c5b-426e-8da5-ca4549d134cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:01.824067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0179e-4c5b-426e-8da5-ca4549d134cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:01.825447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0179e-4c5b-426e-8da5-ca4549d134cf None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:01.835143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 232/449] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:01.840699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0179e-4c5b-426e-8da5-ca4549d134cf None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:01.853496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:01.923289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507c827e-c7fc-4701-8870-4b4f10263db9 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:01.939750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0179e-4c5b-426e-8da5-ca4549d134cf None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:01.957447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c97e5d-a358-483f-bcc4-15bcc12c9acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:01.958084 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c97e5d-a358-483f-bcc4-15bcc12c9acb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.958355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c97e5d-a358-483f-bcc4-15bcc12c9acb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.958625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c97e5d-a358-483f-bcc4-15bcc12c9acb None admin] PATH_INFO: `/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.959733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c97e5d-a358-483f-bcc4-15bcc12c9acb None admin] RBAC: Authorizing `identity:get_group(group_id=c3893fedc58b4b7fa399c12d2c3afcb6)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:01.961186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c97e5d-a358-483f-bcc4-15bcc12c9acb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:01.962737 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 218/450] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:04:01 2022] GET /identity/v3/groups/c3893fedc58b4b7fa399c12d2c3afcb6 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:01.968472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c97e5d-a358-483f-bcc4-15bcc12c9acb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:01.973088 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be667e48-d781-4065-9505-a8e9ccc9547c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:01.973695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be667e48-d781-4065-9505-a8e9ccc9547c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:01.973976 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be667e48-d781-4065-9505-a8e9ccc9547c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:01.974252 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be667e48-d781-4065-9505-a8e9ccc9547c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:01.989387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:02.042099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be667e48-d781-4065-9505-a8e9ccc9547c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:02.043549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be667e48-d781-4065-9505-a8e9ccc9547c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:02.063189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:02.066460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c97e5d-a358-483f-bcc4-15bcc12c9acb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:02.099610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d07f2-090c-4ffc-8ec6-7b19b9d9ad12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:02.100161 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d07f2-090c-4ffc-8ec6-7b19b9d9ad12 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:02.100434 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d07f2-090c-4ffc-8ec6-7b19b9d9ad12 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:02.100701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d07f2-090c-4ffc-8ec6-7b19b9d9ad12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:02.101415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d07f2-090c-4ffc-8ec6-7b19b9d9ad12 None admin] RBAC: Adding query filter params (group.id=c3893fedc58b4b7fa399c12d2c3afcb6, role.id=f8c4be33fa2f4e69a3de71a3c4d7b179, scope.project.id=225b5e655135449ab5cb4e25c5c9fca4) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:02.101715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d07f2-090c-4ffc-8ec6-7b19b9d9ad12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:02.103118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d07f2-090c-4ffc-8ec6-7b19b9d9ad12 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:02.108640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 219/451] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 7 16:04:01 2022] GET /identity/v3/role_assignments?group.id=c3893fedc58b4b7fa399c12d2c3afcb6&scope.project.id=225b5e655135449ab5cb4e25c5c9fca4&role.id=f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:02.145006 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 233/452] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 16:04:02.150999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d07f2-090c-4ffc-8ec6-7b19b9d9ad12 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:02.255506 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d07f2-090c-4ffc-8ec6-7b19b9d9ad12 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:02.290333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779c81c-20a3-4f6f-b040-bba334d9f966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:02.290973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779c81c-20a3-4f6f-b040-bba334d9f966 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:02.291252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779c81c-20a3-4f6f-b040-bba334d9f966 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:02.291534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779c81c-20a3-4f6f-b040-bba334d9f966 None admin] PATH_INFO: `/v3/projects/4f97db8bc2c2455d85720c36a1ed180d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:02.292690 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779c81c-20a3-4f6f-b040-bba334d9f966 None admin] RBAC: Authorizing `identity:get_project(project_id=4f97db8bc2c2455d85720c36a1ed180d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:02.294124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779c81c-20a3-4f6f-b040-bba334d9f966 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:02.295964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 220/453] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:04:02 2022] GET /identity/v3/projects/4f97db8bc2c2455d85720c36a1ed180d => generated 344 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:02.302226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be667e48-d781-4065-9505-a8e9ccc9547c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:02.350778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:02.406701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be667e48-d781-4065-9505-a8e9ccc9547c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:02.442926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8620ea3-7d71-4ebd-b99e-d721b3528367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:02.443557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8620ea3-7d71-4ebd-b99e-d721b3528367 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:02.443875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8620ea3-7d71-4ebd-b99e-d721b3528367 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:02.444149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8620ea3-7d71-4ebd-b99e-d721b3528367 None admin] PATH_INFO: `/v3/groups/b527ebb2c54c4da7ae589da3c6283f65` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:02.445262 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8620ea3-7d71-4ebd-b99e-d721b3528367 None admin] RBAC: Authorizing `identity:get_group(group_id=b527ebb2c54c4da7ae589da3c6283f65)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:02.446690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8620ea3-7d71-4ebd-b99e-d721b3528367 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:02.448474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 234/454] 10.222.0.15 () {56 vars in 1180 bytes} [Wed Sep 7 16:04:02 2022] GET /identity/v3/groups/b527ebb2c54c4da7ae589da3c6283f65 => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:02.454157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779c81c-20a3-4f6f-b040-bba334d9f966 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:02.484789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:02.551736 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779c81c-20a3-4f6f-b040-bba334d9f966 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:02.584662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aabd01-ec06-487a-9aa8-d78a2378ed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:02.585260 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aabd01-ec06-487a-9aa8-d78a2378ed9d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:02.585538 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aabd01-ec06-487a-9aa8-d78a2378ed9d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:02.585811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aabd01-ec06-487a-9aa8-d78a2378ed9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:02.586528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aabd01-ec06-487a-9aa8-d78a2378ed9d None admin] RBAC: Adding query filter params (group.id=b527ebb2c54c4da7ae589da3c6283f65, role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=4f97db8bc2c2455d85720c36a1ed180d) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:02.586834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aabd01-ec06-487a-9aa8-d78a2378ed9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:02.588295 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aabd01-ec06-487a-9aa8-d78a2378ed9d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:02.593815 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 221/455] 10.222.0.15 () {56 vars in 1376 bytes} [Wed Sep 7 16:04:02 2022] GET /identity/v3/role_assignments?group.id=b527ebb2c54c4da7ae589da3c6283f65&scope.project.id=4f97db8bc2c2455d85720c36a1ed180d&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:05.155267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c426d2-c3bf-47de-8b86-ab64dc8e8ea4 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:05.155841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c426d2-c3bf-47de-8b86-ab64dc8e8ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:05.155841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c426d2-c3bf-47de-8b86-ab64dc8e8ea4 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:05.157793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 235/456] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:05.163665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11896d37-7402-4855-9f63-0eb5689b8cd8 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:05.164214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11896d37-7402-4855-9f63-0eb5689b8cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:05.164288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11896d37-7402-4855-9f63-0eb5689b8cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:05.184182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-11896d37-7402-4855-9f63-0eb5689b8cd8 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:05.185401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-11896d37-7402-4855-9f63-0eb5689b8cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:05.271053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd017d-b0df-4756-ad46-37101381e65b None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:05.271342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd017d-b0df-4756-ad46-37101381e65b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:05.271521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd017d-b0df-4756-ad46-37101381e65b None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:05.273136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 236/457] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:05.277850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21feb616-2d5e-41aa-be63-28a05677fc28 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:05.278128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21feb616-2d5e-41aa-be63-28a05677fc28 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:05.278402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21feb616-2d5e-41aa-be63-28a05677fc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:05.281218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 222/458] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:05.312988 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-21feb616-2d5e-41aa-be63-28a05677fc28 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:05.314781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-21feb616-2d5e-41aa-be63-28a05677fc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:05.415666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 237/459] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:05.527549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61362a24-f2f2-4ccd-af6f-06c6dfb53ddd None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:05.527866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61362a24-f2f2-4ccd-af6f-06c6dfb53ddd None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:05.528051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61362a24-f2f2-4ccd-af6f-06c6dfb53ddd None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:05.529697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 223/460] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:05.535739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0c2de-1f2a-48f0-be37-9cc78f156aa7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:05.535739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0c2de-1f2a-48f0-be37-9cc78f156aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:05.535739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0c2de-1f2a-48f0-be37-9cc78f156aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:05.561818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-d3b0c2de-1f2a-48f0-be37-9cc78f156aa7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:05.563356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d3b0c2de-1f2a-48f0-be37-9cc78f156aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:05.607400 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61362a24-f2f2-4ccd-af6f-06c6dfb53ddd None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:05.616198 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:05.651869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 238/461] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:05.680186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:05.724824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0c2de-1f2a-48f0-be37-9cc78f156aa7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:05.826657 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61362a24-f2f2-4ccd-af6f-06c6dfb53ddd None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:05.883813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40fe0a-e03b-4767-ae86-a804e2cc8c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:05.883813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40fe0a-e03b-4767-ae86-a804e2cc8c00 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:05.883813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40fe0a-e03b-4767-ae86-a804e2cc8c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:05.883813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40fe0a-e03b-4767-ae86-a804e2cc8c00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:05.903872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40fe0a-e03b-4767-ae86-a804e2cc8c00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:05.905325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40fe0a-e03b-4767-ae86-a804e2cc8c00 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:05.911854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0c2de-1f2a-48f0-be37-9cc78f156aa7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-6a40fe0a-e03b-4767-ae86-a804e2cc8c00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:05.928010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:05.928948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:05.930324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:05.930324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:05.930324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:05.930324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 224/462] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:05.936713 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40fe0a-e03b-4767-ae86-a804e2cc8c00 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:05.960545 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c52284-45a4-48c2-954f-772d4f2c3995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:05.961191 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c52284-45a4-48c2-954f-772d4f2c3995 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:05.961473 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c52284-45a4-48c2-954f-772d4f2c3995 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:05.961894 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c52284-45a4-48c2-954f-772d4f2c3995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:05.979914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c52284-45a4-48c2-954f-772d4f2c3995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:05.981438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c52284-45a4-48c2-954f-772d4f2c3995 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-42c52284-45a4-48c2-954f-772d4f2c3995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:06.009891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:06.010773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:06.011631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:06.011631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:06.011631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:06.013619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 239/463] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:06.014269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c52284-45a4-48c2-954f-772d4f2c3995 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:06.058926 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:06.061771 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:06.088122 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40fe0a-e03b-4767-ae86-a804e2cc8c00 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:06.129537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f26371-b40a-450b-b524-b75c6ac7c1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:06.130244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26371-b40a-450b-b524-b75c6ac7c1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:06.130524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26371-b40a-450b-b524-b75c6ac7c1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:06.130792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f26371-b40a-450b-b524-b75c6ac7c1a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:06.131433 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f26371-b40a-450b-b524-b75c6ac7c1a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:06.131729 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f26371-b40a-450b-b524-b75c6ac7c1a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:06.133144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f26371-b40a-450b-b524-b75c6ac7c1a4 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:06.148050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 225/464] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:06.148892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f26371-b40a-450b-b524-b75c6ac7c1a4 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:06.158419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c52284-45a4-48c2-954f-772d4f2c3995 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:06.196316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e1772-9816-4a0e-b616-cff3f34d967c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:06.196954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e1772-9816-4a0e-b616-cff3f34d967c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:06.197238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e1772-9816-4a0e-b616-cff3f34d967c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:06.197517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e1772-9816-4a0e-b616-cff3f34d967c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:06.211203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e1772-9816-4a0e-b616-cff3f34d967c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:06.212954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e1772-9816-4a0e-b616-cff3f34d967c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-cd2e1772-9816-4a0e-b616-cff3f34d967c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:06.228004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:06.228906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:06.229743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:06.229743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:06.229743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:06.229743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 240/465] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:06.229964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e1772-9816-4a0e-b616-cff3f34d967c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:06.263236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f26371-b40a-450b-b524-b75c6ac7c1a4 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:06.299954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684feaf-a611-4081-a0cb-2d85eb623119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:06.300648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684feaf-a611-4081-a0cb-2d85eb623119 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:06.300932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684feaf-a611-4081-a0cb-2d85eb623119 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:06.301222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684feaf-a611-4081-a0cb-2d85eb623119 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:06.301850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684feaf-a611-4081-a0cb-2d85eb623119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:06.302166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684feaf-a611-4081-a0cb-2d85eb623119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:06.303886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684feaf-a611-4081-a0cb-2d85eb623119 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:06.310742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:06.316429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 226/466] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:06.317514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c12b-97c3-48e3-be54-b07c5e74e440 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:06.317801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c12b-97c3-48e3-be54-b07c5e74e440 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:06.318076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c12b-97c3-48e3-be54-b07c5e74e440 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:06.319340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 227/467] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:06.323395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:06.324181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034c12b-97c3-48e3-be54-b07c5e74e440 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:06.343531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e1772-9816-4a0e-b616-cff3f34d967c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:06.377173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6033ec-0239-4838-a126-cdf71ce10f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:06.377807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6033ec-0239-4838-a126-cdf71ce10f70 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:06.378100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6033ec-0239-4838-a126-cdf71ce10f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:06.378499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6033ec-0239-4838-a126-cdf71ce10f70 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:06.379256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6033ec-0239-4838-a126-cdf71ce10f70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:06.380795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6033ec-0239-4838-a126-cdf71ce10f70 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:06.435838 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034c12b-97c3-48e3-be54-b07c5e74e440 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:06.467793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a601ef-07f0-47d7-b3a1-37c2df40b340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:06.468396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a601ef-07f0-47d7-b3a1-37c2df40b340 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:06.468670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a601ef-07f0-47d7-b3a1-37c2df40b340 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:06.468943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a601ef-07f0-47d7-b3a1-37c2df40b340 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:06.469575 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a601ef-07f0-47d7-b3a1-37c2df40b340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:06.470404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a601ef-07f0-47d7-b3a1-37c2df40b340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:06.471751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a601ef-07f0-47d7-b3a1-37c2df40b340 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:06.483674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 241/468] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:04:06 2022] POST /identity/v3/users => generated 246 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:04:06.489216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 228/469] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:06.489216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db9506-42d5-4b05-87a3-3a93a3e017fc None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:06.489216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db9506-42d5-4b05-87a3-3a93a3e017fc None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:06.489216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db9506-42d5-4b05-87a3-3a93a3e017fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:06.490641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6033ec-0239-4838-a126-cdf71ce10f70 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:06.515638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-d5db9506-42d5-4b05-87a3-3a93a3e017fc None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:06.517301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d5db9506-42d5-4b05-87a3-3a93a3e017fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:06.593742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:06.610309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 229/470] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:06.611919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db9506-42d5-4b05-87a3-3a93a3e017fc None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:06.639215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6033ec-0239-4838-a126-cdf71ce10f70 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:06.687625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f4544-c043-4a99-b88a-d2ec096a5bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:06.688289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f4544-c043-4a99-b88a-d2ec096a5bed None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:06.688590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f4544-c043-4a99-b88a-d2ec096a5bed None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:06.688869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f4544-c043-4a99-b88a-d2ec096a5bed None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:06.689512 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f4544-c043-4a99-b88a-d2ec096a5bed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:06.690950 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f4544-c043-4a99-b88a-d2ec096a5bed None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:06.701712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:06.741552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db9506-42d5-4b05-87a3-3a93a3e017fc None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:06.777860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 242/471] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:04:06 2022] POST /identity/v3/users => generated 248 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:04:06.785689 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774106a2-8f1d-4ecf-acf7-63435e9ef70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:06.786694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774106a2-8f1d-4ecf-acf7-63435e9ef70f None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:06.787454 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774106a2-8f1d-4ecf-acf7-63435e9ef70f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:06.788014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774106a2-8f1d-4ecf-acf7-63435e9ef70f None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:06.789101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774106a2-8f1d-4ecf-acf7-63435e9ef70f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:06.790853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774106a2-8f1d-4ecf-acf7-63435e9ef70f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:06.914683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 230/472] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:04:06 2022] POST /identity/v3/users => generated 248 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:04:06.933660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f4544-c043-4a99-b88a-d2ec096a5bed None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:07.059161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f4544-c043-4a99-b88a-d2ec096a5bed None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:07.098253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbd2d7-c7f7-441e-a66d-fa1f77387df2 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:07.098541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbd2d7-c7f7-441e-a66d-fa1f77387df2 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:07.098822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcbd2d7-c7f7-441e-a66d-fa1f77387df2 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:07.099661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a99149-b5db-467f-8eba-907216170ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:07.100330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a99149-b5db-467f-8eba-907216170ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:07.100532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 231/473] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:04:07.100631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a99149-b5db-467f-8eba-907216170ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:07.100871 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a99149-b5db-467f-8eba-907216170ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:07.107679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e219c-d97c-436a-9426-978a7949e9c4 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:07.108062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e219c-d97c-436a-9426-978a7949e9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:07.108338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e219c-d97c-436a-9426-978a7949e9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:07.113622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a99149-b5db-467f-8eba-907216170ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:07.115143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a99149-b5db-467f-8eba-907216170ce4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-f2a99149-b5db-467f-8eba-907216170ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:07.132807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:07.133972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:07.135119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:07.135119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:07.135119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:07.135119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 243/474] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:07.136533 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-471e219c-d97c-436a-9426-978a7949e9c4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:07.138829 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-471e219c-d97c-436a-9426-978a7949e9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:07.140323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a99149-b5db-467f-8eba-907216170ce4 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:07.191060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:07.201377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:07.234619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 232/475] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:07.268426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a99149-b5db-467f-8eba-907216170ce4 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:07.310659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50143ec1-c89f-45d9-849f-a2e72976482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:07.311313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143ec1-c89f-45d9-849f-a2e72976482d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:07.311616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143ec1-c89f-45d9-849f-a2e72976482d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:07.311933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50143ec1-c89f-45d9-849f-a2e72976482d None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:07.312559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50143ec1-c89f-45d9-849f-a2e72976482d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:07.312860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50143ec1-c89f-45d9-849f-a2e72976482d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:07.314767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50143ec1-c89f-45d9-849f-a2e72976482d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:07.327732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 244/476] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:07.333931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e219c-d97c-436a-9426-978a7949e9c4 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:07.447369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e219c-d97c-436a-9426-978a7949e9c4 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:07.491362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1a6be-e567-49a9-8a76-acb15cba0657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:07.492442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1a6be-e567-49a9-8a76-acb15cba0657 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:07.492896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1a6be-e567-49a9-8a76-acb15cba0657 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:07.493335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db1a6be-e567-49a9-8a76-acb15cba0657 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:07.494145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1a6be-e567-49a9-8a76-acb15cba0657 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:07.495905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db1a6be-e567-49a9-8a76-acb15cba0657 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:07.528239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50143ec1-c89f-45d9-849f-a2e72976482d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:07.620153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 233/477] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:04:07 2022] POST /identity/v3/users => generated 249 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:04:07.647274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50143ec1-c89f-45d9-849f-a2e72976482d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:07.649440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:07.686571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a757d-a0f9-4d65-b74f-b82cbcae3806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:07.687500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a757d-a0f9-4d65-b74f-b82cbcae3806 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:07.687992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a757d-a0f9-4d65-b74f-b82cbcae3806 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:07.688531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a757d-a0f9-4d65-b74f-b82cbcae3806 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:07.689504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a757d-a0f9-4d65-b74f-b82cbcae3806 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:07.691088 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a757d-a0f9-4d65-b74f-b82cbcae3806 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:07.740135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.sql.core [None req-2e9a757d-a0f9-4d65-b74f-b82cbcae3806 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 16:04:07.740135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 16:04:07.740135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [parameters: {'id': 'e4316a36ad9a4f4d9aad5befd56b431e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 16:04:07.740135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81775) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-2e9a757d-a0f9-4d65-b74f-b82cbcae3806 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application conn.query(q) Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application result.read() Sep 07 16:04:07.761350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return ref.to_dict() Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 16:04:07.768130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application next(self.gen) Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application yield resource Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application next(self.gen) Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise self.value Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application session.commit() Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 16:04:07.769231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self.session.flush() Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application compat.raise_( Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 16:04:07.770249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 16:04:07.771307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application conn.query(q) Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application result.read() Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 16:04:07.772203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [parameters: {'id': 'e4316a36ad9a4f4d9aad5befd56b431e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:07.773113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 16:04:07.774113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:07.775107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 245/478] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Sep 7 16:04:07 2022] POST /identity/v3/roles => generated 144 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:07.775181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1a6be-e567-49a9-8a76-acb15cba0657 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:07.837309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:07.913404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db1a6be-e567-49a9-8a76-acb15cba0657 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:07.949628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756f3d5-6388-44f6-ab1d-aa0775d59442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:07.950258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756f3d5-6388-44f6-ab1d-aa0775d59442 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:07.950534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756f3d5-6388-44f6-ab1d-aa0775d59442 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:07.950812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756f3d5-6388-44f6-ab1d-aa0775d59442 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:07.961858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756f3d5-6388-44f6-ab1d-aa0775d59442 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-3756f3d5-6388-44f6-ab1d-aa0775d59442 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:07.967801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:07.968665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:07.969453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 234/479] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:04:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:07.974396 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a757d-a0f9-4d65-b74f-b82cbcae3806 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:08.139712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a757d-a0f9-4d65-b74f-b82cbcae3806 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:08.181965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76186a21-b337-4a14-a66c-fb285f2ce322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:08.182581 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186a21-b337-4a14-a66c-fb285f2ce322 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:08.182858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186a21-b337-4a14-a66c-fb285f2ce322 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:08.183131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76186a21-b337-4a14-a66c-fb285f2ce322 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:08.183809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76186a21-b337-4a14-a66c-fb285f2ce322 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:08.184114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76186a21-b337-4a14-a66c-fb285f2ce322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:08.185484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76186a21-b337-4a14-a66c-fb285f2ce322 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:08.194755 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 246/480] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:09.189876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725a5d9-2cf4-44a3-a45c-d1c65ed055a7 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:09.190891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725a5d9-2cf4-44a3-a45c-d1c65ed055a7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:09.191368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6725a5d9-2cf4-44a3-a45c-d1c65ed055a7 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:09.193007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 235/481] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:09.198261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6b4d1-85d9-41bb-9087-f49f2119e653 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:09.199112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6b4d1-85d9-41bb-9087-f49f2119e653 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:09.199620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6b4d1-85d9-41bb-9087-f49f2119e653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:09.226394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-d5a6b4d1-85d9-41bb-9087-f49f2119e653 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:09.227930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d5a6b4d1-85d9-41bb-9087-f49f2119e653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:09.255806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:09.312343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 247/482] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:09.691193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725a5d9-2cf4-44a3-a45c-d1c65ed055a7 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:09.725248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4646845-82e3-40a0-b4eb-8152c3e9eede None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:09.725565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4646845-82e3-40a0-b4eb-8152c3e9eede None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:09.725818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4646845-82e3-40a0-b4eb-8152c3e9eede None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:09.727415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 248/483] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:09.738119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a795454-5c39-4ec4-bb87-5865ba4aac30 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:09.738406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a795454-5c39-4ec4-bb87-5865ba4aac30 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:09.738689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a795454-5c39-4ec4-bb87-5865ba4aac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:09.769204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6a795454-5c39-4ec4-bb87-5865ba4aac30 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:09.770536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6a795454-5c39-4ec4-bb87-5865ba4aac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:09.806315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6725a5d9-2cf4-44a3-a45c-d1c65ed055a7 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:09.812074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:09.840673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 249/484] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:09.842052 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf56a3a-fec3-41ff-be93-652ae5bd3b44 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:09.842342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf56a3a-fec3-41ff-be93-652ae5bd3b44 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:09.842620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf56a3a-fec3-41ff-be93-652ae5bd3b44 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:09.844179 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 250/485] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:09.852495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a11d4d-a5b1-4f43-b097-41ce80182264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:09.853358 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a11d4d-a5b1-4f43-b097-41ce80182264 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:09.853760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a11d4d-a5b1-4f43-b097-41ce80182264 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:09.854206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a11d4d-a5b1-4f43-b097-41ce80182264 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:09.857865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd7448-ecce-4840-8fc6-9a222f372a00 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:09.858157 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd7448-ecce-4840-8fc6-9a222f372a00 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:09.858435 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd7448-ecce-4840-8fc6-9a222f372a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:09.870585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a11d4d-a5b1-4f43-b097-41ce80182264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-52a11d4d-a5b1-4f43-b097-41ce80182264 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:09.881585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:09.882465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:09.883303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:09.886134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 236/486] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:09.892499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a11d4d-a5b1-4f43-b097-41ce80182264 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:09.896849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-53dd7448-ecce-4840-8fc6-9a222f372a00 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:09.899146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-53dd7448-ecce-4840-8fc6-9a222f372a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:10.037528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 251/487] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:10.070630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a11d4d-a5b1-4f43-b097-41ce80182264 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:10.120564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3de2a9-146e-44ca-b466-115b18328d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:10.121583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3de2a9-146e-44ca-b466-115b18328d3b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:10.122075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3de2a9-146e-44ca-b466-115b18328d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:10.122546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3de2a9-146e-44ca-b466-115b18328d3b None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:10.123390 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3de2a9-146e-44ca-b466-115b18328d3b None admin] RBAC: Adding query filter params (name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:10.123906 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3de2a9-146e-44ca-b466-115b18328d3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:10.125446 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3de2a9-146e-44ca-b466-115b18328d3b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:10.137610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 237/488] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:10.143395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd7448-ecce-4840-8fc6-9a222f372a00 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:10.166364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:10.172228 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3de2a9-146e-44ca-b466-115b18328d3b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:10.188702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:10.308930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3de2a9-146e-44ca-b466-115b18328d3b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:10.317837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd7448-ecce-4840-8fc6-9a222f372a00 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:10.348420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad52941-9060-471e-85a3-bc828237903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:10.349488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad52941-9060-471e-85a3-bc828237903a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:10.350089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad52941-9060-471e-85a3-bc828237903a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:10.350564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad52941-9060-471e-85a3-bc828237903a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:10.362105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad52941-9060-471e-85a3-bc828237903a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:10.362274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70da2b-254d-460a-92df-8b66467628a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:10.362866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70da2b-254d-460a-92df-8b66467628a1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:10.363149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70da2b-254d-460a-92df-8b66467628a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:10.363427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70da2b-254d-460a-92df-8b66467628a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-9ad52941-9060-471e-85a3-bc828237903a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:10.369124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:10.370083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:10.370884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:10.372975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 238/489] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:10.373760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad52941-9060-471e-85a3-bc828237903a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:10.378031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70da2b-254d-460a-92df-8b66467628a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:10.379874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70da2b-254d-460a-92df-8b66467628a1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-cd70da2b-254d-460a-92df-8b66467628a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:10.396257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:10.397180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:10.398031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:10.398031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:10.398031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:10.400190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 252/490] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:10.401232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70da2b-254d-460a-92df-8b66467628a1 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:10.464943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:10.495878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:10.502026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad52941-9060-471e-85a3-bc828237903a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:10.534410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70da2b-254d-460a-92df-8b66467628a1 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:10.551498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbf197-3df0-4c09-965c-05dfaa5fd7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:10.552325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf197-3df0-4c09-965c-05dfaa5fd7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:10.552607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf197-3df0-4c09-965c-05dfaa5fd7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:10.552880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf197-3df0-4c09-965c-05dfaa5fd7e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:10.565136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abbf197-3df0-4c09-965c-05dfaa5fd7e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-3abbf197-3df0-4c09-965c-05dfaa5fd7e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:10.570601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:10.571490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:10.572316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 239/491] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:10.572965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbf197-3df0-4c09-965c-05dfaa5fd7e8 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:10.578791 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67489e23-a347-454d-8fb7-b7966a167b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:10.579411 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67489e23-a347-454d-8fb7-b7966a167b4a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:10.579692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67489e23-a347-454d-8fb7-b7966a167b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:10.580040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67489e23-a347-454d-8fb7-b7966a167b4a None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:10.580707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67489e23-a347-454d-8fb7-b7966a167b4a None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:10.581013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67489e23-a347-454d-8fb7-b7966a167b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:10.582390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67489e23-a347-454d-8fb7-b7966a167b4a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:10.590926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 253/492] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:10.592279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d0783-dcc3-42ce-9629-7d1d3feec894 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:10.592566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d0783-dcc3-42ce-9629-7d1d3feec894 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:10.592839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d0783-dcc3-42ce-9629-7d1d3feec894 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:10.598025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 254/493] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:10.598025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d0783-dcc3-42ce-9629-7d1d3feec894 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:10.690421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbf197-3df0-4c09-965c-05dfaa5fd7e8 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:10.703401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d0783-dcc3-42ce-9629-7d1d3feec894 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:10.721489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:10.726398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16d3b8-2443-41fc-88ac-8e42ebac228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:10.727118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16d3b8-2443-41fc-88ac-8e42ebac228c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:10.727742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16d3b8-2443-41fc-88ac-8e42ebac228c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:10.727881 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16d3b8-2443-41fc-88ac-8e42ebac228c None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:10.728486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16d3b8-2443-41fc-88ac-8e42ebac228c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:10.728766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16d3b8-2443-41fc-88ac-8e42ebac228c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:10.730261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16d3b8-2443-41fc-88ac-8e42ebac228c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:10.736298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:10.739055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e2f8d-2137-43be-8240-bcc0170074d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:10.739666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e2f8d-2137-43be-8240-bcc0170074d9 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:10.739975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e2f8d-2137-43be-8240-bcc0170074d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:10.740248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e2f8d-2137-43be-8240-bcc0170074d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:10.740912 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e2f8d-2137-43be-8240-bcc0170074d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:10.741217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e2f8d-2137-43be-8240-bcc0170074d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:10.742618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e2f8d-2137-43be-8240-bcc0170074d9 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:10.747994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 240/494] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:10.748661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16d3b8-2443-41fc-88ac-8e42ebac228c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:10.756379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 255/495] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:10.756379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63218178-fb4f-4a62-bd9f-1328dc502c51 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:10.756379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63218178-fb4f-4a62-bd9f-1328dc502c51 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:10.756379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63218178-fb4f-4a62-bd9f-1328dc502c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:10.771384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-63218178-fb4f-4a62-bd9f-1328dc502c51 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:10.772808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-63218178-fb4f-4a62-bd9f-1328dc502c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:10.847562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 256/496] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:10.847562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63218178-fb4f-4a62-bd9f-1328dc502c51 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:10.869996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16d3b8-2443-41fc-88ac-8e42ebac228c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:10.916591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aae3ad-d513-4f07-8e18-23b1f585fadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:10.917989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aae3ad-d513-4f07-8e18-23b1f585fadc None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:10.918426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aae3ad-d513-4f07-8e18-23b1f585fadc None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:10.918837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aae3ad-d513-4f07-8e18-23b1f585fadc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:10.933825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aae3ad-d513-4f07-8e18-23b1f585fadc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:10.936048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aae3ad-d513-4f07-8e18-23b1f585fadc None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-33aae3ad-d513-4f07-8e18-23b1f585fadc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:10.959279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:10.960844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:10.961868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:10.961868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:10.961868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:10.967702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 241/497] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:10.968919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aae3ad-d513-4f07-8e18-23b1f585fadc None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:10.983433 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63218178-fb4f-4a62-bd9f-1328dc502c51 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:11.010324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:11.028810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56bd1c-6588-41e4-90ed-7180fcee6d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:11.029586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bd1c-6588-41e4-90ed-7180fcee6d80 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.029963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bd1c-6588-41e4-90ed-7180fcee6d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.030335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bd1c-6588-41e4-90ed-7180fcee6d80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.039055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56bd1c-6588-41e4-90ed-7180fcee6d80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:11.041114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56bd1c-6588-41e4-90ed-7180fcee6d80 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:11.048063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-7d56bd1c-6588-41e4-90ed-7180fcee6d80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:11.055751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:11.057161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:11.058336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:11.058336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:11.058336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:11.058336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:11.058336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:11.058336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:11.058336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:11.058336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:11.058336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:11.058336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:11.064385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 257/498] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 7 16:04:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:11.064385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56bd1c-6588-41e4-90ed-7180fcee6d80 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:11.121259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aae3ad-d513-4f07-8e18-23b1f585fadc None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:11.153714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306cad4-7d4e-493a-b29f-7ebd6fb13d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:11.154331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306cad4-7d4e-493a-b29f-7ebd6fb13d83 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.154609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306cad4-7d4e-493a-b29f-7ebd6fb13d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.154883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306cad4-7d4e-493a-b29f-7ebd6fb13d83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.166014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306cad4-7d4e-493a-b29f-7ebd6fb13d83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:11.167489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306cad4-7d4e-493a-b29f-7ebd6fb13d83 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:11.169683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56bd1c-6588-41e4-90ed-7180fcee6d80 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-2306cad4-7d4e-493a-b29f-7ebd6fb13d83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:11.183656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:11.191692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:11.192804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:11.192804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:11.192804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:11.192804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 242/499] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:11.192804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306cad4-7d4e-493a-b29f-7ebd6fb13d83 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:11.207249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc3d10-9f0c-42db-8269-33992ea17778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:11.207881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3d10-9f0c-42db-8269-33992ea17778 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.208158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3d10-9f0c-42db-8269-33992ea17778 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.208424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3d10-9f0c-42db-8269-33992ea17778 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.209048 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3d10-9f0c-42db-8269-33992ea17778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:11.209345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3d10-9f0c-42db-8269-33992ea17778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:11.210768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3d10-9f0c-42db-8269-33992ea17778 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:11.223998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 258/500] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:11.225164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73298cc0-aefd-4e4a-9fd2-d9c6d4881eef None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.225444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73298cc0-aefd-4e4a-9fd2-d9c6d4881eef None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.225712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73298cc0-aefd-4e4a-9fd2-d9c6d4881eef None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.227021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 259/501] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:11.231546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73298cc0-aefd-4e4a-9fd2-d9c6d4881eef None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:11.278591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:11.286415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:11.291983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306cad4-7d4e-493a-b29f-7ebd6fb13d83 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:11.324850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9bb9b-26f9-4ffb-bd87-a5e6178398f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:11.325463 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9bb9b-26f9-4ffb-bd87-a5e6178398f7 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.325741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9bb9b-26f9-4ffb-bd87-a5e6178398f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.326012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9bb9b-26f9-4ffb-bd87-a5e6178398f7 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.326729 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9bb9b-26f9-4ffb-bd87-a5e6178398f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:11.327028 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9bb9b-26f9-4ffb-bd87-a5e6178398f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:11.328484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9bb9b-26f9-4ffb-bd87-a5e6178398f7 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:11.332626 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73298cc0-aefd-4e4a-9fd2-d9c6d4881eef None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:11.369753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 243/502] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 7 16:04:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:11.369928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e6f2d-c19b-4a19-94bc-03431e72ec36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:11.370033 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9bb9b-26f9-4ffb-bd87-a5e6178398f7 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:11.370139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6f2d-c19b-4a19-94bc-03431e72ec36 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.370139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6f2d-c19b-4a19-94bc-03431e72ec36 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.370139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e6f2d-c19b-4a19-94bc-03431e72ec36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.376662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e6f2d-c19b-4a19-94bc-03431e72ec36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-903e6f2d-c19b-4a19-94bc-03431e72ec36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:11.381381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:11.382446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 260/503] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:11.383247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e6f2d-c19b-4a19-94bc-03431e72ec36 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:11.468868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9bb9b-26f9-4ffb-bd87-a5e6178398f7 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:11.483364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e6f2d-c19b-4a19-94bc-03431e72ec36 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:11.486158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:11.505472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4b413a-91f5-4eb8-840d-e5afdc167c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:11.506084 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b413a-91f5-4eb8-840d-e5afdc167c5c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.506367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b413a-91f5-4eb8-840d-e5afdc167c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.506643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b413a-91f5-4eb8-840d-e5afdc167c5c None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.507471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b413a-91f5-4eb8-840d-e5afdc167c5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:11.508078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b413a-91f5-4eb8-840d-e5afdc167c5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:11.510252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4b413a-91f5-4eb8-840d-e5afdc167c5c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:11.521999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 244/504] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:11.524232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a2cf6-696e-428a-ad55-7a6f886609a0 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.524679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a2cf6-696e-428a-ad55-7a6f886609a0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.524962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a2cf6-696e-428a-ad55-7a6f886609a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.532558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efae17f-f841-41f8-ba70-7986478ce585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:11.533196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efae17f-f841-41f8-ba70-7986478ce585 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.533480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efae17f-f841-41f8-ba70-7986478ce585 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.533754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efae17f-f841-41f8-ba70-7986478ce585 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.541079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efae17f-f841-41f8-ba70-7986478ce585 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:11.542528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efae17f-f841-41f8-ba70-7986478ce585 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:11.545841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7d2a2cf6-696e-428a-ad55-7a6f886609a0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:11.547087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7d2a2cf6-696e-428a-ad55-7a6f886609a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-6efae17f-f841-41f8-ba70-7986478ce585 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:11.554603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:11.555700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 261/505] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:11.557139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efae17f-f841-41f8-ba70-7986478ce585 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:11.561899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:11.618968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 245/506] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:11.619860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a2cf6-696e-428a-ad55-7a6f886609a0 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:11.658311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efae17f-f841-41f8-ba70-7986478ce585 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:11.693539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92cfe8-dbc9-4e66-a267-342955f52cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:11.694148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92cfe8-dbc9-4e66-a267-342955f52cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.694422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92cfe8-dbc9-4e66-a267-342955f52cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.694705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92cfe8-dbc9-4e66-a267-342955f52cb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.725740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:11.738897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a2cf6-696e-428a-ad55-7a6f886609a0 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:11.763866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92cfe8-dbc9-4e66-a267-342955f52cb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:11.765368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92cfe8-dbc9-4e66-a267-342955f52cb2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:11.774772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bb547-d875-48af-8e1e-0067948a0cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:11.775378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bb547-d875-48af-8e1e-0067948a0cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.775658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bb547-d875-48af-8e1e-0067948a0cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.776072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bb547-d875-48af-8e1e-0067948a0cd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.776734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bb547-d875-48af-8e1e-0067948a0cd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:11.777053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bb547-d875-48af-8e1e-0067948a0cd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:11.778393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bb547-d875-48af-8e1e-0067948a0cd1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:11.787064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 246/507] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:11.787970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bb547-d875-48af-8e1e-0067948a0cd1 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:11.800734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:11.873470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 262/508] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:11.874342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92cfe8-dbc9-4e66-a267-342955f52cb2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:11.904959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bb547-d875-48af-8e1e-0067948a0cd1 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:11.934263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:11.939230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cca49f-a1dd-4cce-8f90-fb607ede7f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:11.939880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cca49f-a1dd-4cce-8f90-fb607ede7f53 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.942962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cca49f-a1dd-4cce-8f90-fb607ede7f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.942962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cca49f-a1dd-4cce-8f90-fb607ede7f53 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.947627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cca49f-a1dd-4cce-8f90-fb607ede7f53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:11.949081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cca49f-a1dd-4cce-8f90-fb607ede7f53 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-c9cca49f-a1dd-4cce-8f90-fb607ede7f53 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:11.961611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:11.972548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 247/509] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:11.973436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cca49f-a1dd-4cce-8f90-fb607ede7f53 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:11.974038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92cfe8-dbc9-4e66-a267-342955f52cb2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:12.007496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e6629-f45c-47ed-940c-4550917d007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:12.008135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e6629-f45c-47ed-940c-4550917d007b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:12.008926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e6629-f45c-47ed-940c-4550917d007b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:12.008926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e6629-f45c-47ed-940c-4550917d007b None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:12.009399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e6629-f45c-47ed-940c-4550917d007b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:12.009705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e6629-f45c-47ed-940c-4550917d007b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:12.011191 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e6629-f45c-47ed-940c-4550917d007b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:12.030919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:12.048244 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 263/510] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 7 16:04:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:12.049061 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e6629-f45c-47ed-940c-4550917d007b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:12.070636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cca49f-a1dd-4cce-8f90-fb607ede7f53 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:12.103930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af4b9b-20a6-44ab-b507-418cedb0fbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:12.104546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af4b9b-20a6-44ab-b507-418cedb0fbd5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:12.104821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af4b9b-20a6-44ab-b507-418cedb0fbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:12.105090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af4b9b-20a6-44ab-b507-418cedb0fbd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:12.115261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af4b9b-20a6-44ab-b507-418cedb0fbd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:12.116717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af4b9b-20a6-44ab-b507-418cedb0fbd5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-68af4b9b-20a6-44ab-b507-418cedb0fbd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:12.130720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:12.136371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:12.137489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:12.137489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:12.137489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:12.137489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 248/511] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:12.137489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af4b9b-20a6-44ab-b507-418cedb0fbd5 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:12.147951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e6629-f45c-47ed-940c-4550917d007b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:12.170063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:12.181233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff794c-b24d-4955-a75b-81f32658519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:12.183869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff794c-b24d-4955-a75b-81f32658519c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:12.183869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff794c-b24d-4955-a75b-81f32658519c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:12.183869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff794c-b24d-4955-a75b-81f32658519c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:12.192647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff794c-b24d-4955-a75b-81f32658519c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:12.194027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff794c-b24d-4955-a75b-81f32658519c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-52ff794c-b24d-4955-a75b-81f32658519c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:12.208704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:12.209735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:12.210820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:12.210820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:12.210820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:12.210820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 264/512] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:12.211113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff794c-b24d-4955-a75b-81f32658519c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:12.235147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af4b9b-20a6-44ab-b507-418cedb0fbd5 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:12.251487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:12.268058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc3ee-fbcc-4627-98e2-977372ae7fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:12.268682 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc3ee-fbcc-4627-98e2-977372ae7fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:12.268960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc3ee-fbcc-4627-98e2-977372ae7fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:12.269235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc3ee-fbcc-4627-98e2-977372ae7fc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:12.279278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddc3ee-fbcc-4627-98e2-977372ae7fc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:12.281052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddc3ee-fbcc-4627-98e2-977372ae7fc1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-94ddc3ee-fbcc-4627-98e2-977372ae7fc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:12.295697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:12.296834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:12.297899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:12.297899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:12.297899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:12.297899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 249/513] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:12.297899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc3ee-fbcc-4627-98e2-977372ae7fc1 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:12.311184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff794c-b24d-4955-a75b-81f32658519c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:12.344238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebd12a-8b71-4b38-a15a-15e940071920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:12.345430 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebd12a-8b71-4b38-a15a-15e940071920 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:12.345710 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebd12a-8b71-4b38-a15a-15e940071920 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:12.345986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebd12a-8b71-4b38-a15a-15e940071920 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:12.346712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebd12a-8b71-4b38-a15a-15e940071920 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:12.347017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebd12a-8b71-4b38-a15a-15e940071920 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:12.348507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebd12a-8b71-4b38-a15a-15e940071920 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:12.364947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 265/514] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 7 16:04:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:12.365728 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebd12a-8b71-4b38-a15a-15e940071920 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:12.396000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:12.402907 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc3ee-fbcc-4627-98e2-977372ae7fc1 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:12.435342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c1de6-4518-40d9-bc6e-4e4941571427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:12.435976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1de6-4518-40d9-bc6e-4e4941571427 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:12.436260 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1de6-4518-40d9-bc6e-4e4941571427 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:12.436535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1de6-4518-40d9-bc6e-4e4941571427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:12.465441 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebd12a-8b71-4b38-a15a-15e940071920 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:12.480305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:12.498000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360afc92-3322-4730-83f4-d6018f3d6cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:12.498606 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360afc92-3322-4730-83f4-d6018f3d6cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:12.498884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360afc92-3322-4730-83f4-d6018f3d6cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:12.499157 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360afc92-3322-4730-83f4-d6018f3d6cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:12.499847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360afc92-3322-4730-83f4-d6018f3d6cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:12.500150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360afc92-3322-4730-83f4-d6018f3d6cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:12.502063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360afc92-3322-4730-83f4-d6018f3d6cd9 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:12.508508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c1de6-4518-40d9-bc6e-4e4941571427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:12.509934 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c1de6-4518-40d9-bc6e-4e4941571427 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:12.513358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 266/515] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:12.514141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360afc92-3322-4730-83f4-d6018f3d6cd9 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:12.608984 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 250/516] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:12.609201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:12.609832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c1de6-4518-40d9-bc6e-4e4941571427 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:12.622222 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360afc92-3322-4730-83f4-d6018f3d6cd9 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:12.656306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a34aa0-300e-4cd5-9edd-39f9860eb0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:12.657041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a34aa0-300e-4cd5-9edd-39f9860eb0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:12.657321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a34aa0-300e-4cd5-9edd-39f9860eb0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:12.657594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a34aa0-300e-4cd5-9edd-39f9860eb0c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:12.658230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a34aa0-300e-4cd5-9edd-39f9860eb0c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:12.658531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a34aa0-300e-4cd5-9edd-39f9860eb0c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:12.659926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a34aa0-300e-4cd5-9edd-39f9860eb0c4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:12.671052 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 267/517] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:12.671851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a34aa0-300e-4cd5-9edd-39f9860eb0c4 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:12.679556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:12.711055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c1de6-4518-40d9-bc6e-4e4941571427 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:12.743855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d51dc-4e8a-40bb-bc3f-24f1e09525dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:12.744393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d51dc-4e8a-40bb-bc3f-24f1e09525dd None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:12.744673 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d51dc-4e8a-40bb-bc3f-24f1e09525dd None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:12.744949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d51dc-4e8a-40bb-bc3f-24f1e09525dd None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:12.745582 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d51dc-4e8a-40bb-bc3f-24f1e09525dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:12.745888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d51dc-4e8a-40bb-bc3f-24f1e09525dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:12.747253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d51dc-4e8a-40bb-bc3f-24f1e09525dd None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:12.759227 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 251/518] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:12.760029 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d51dc-4e8a-40bb-bc3f-24f1e09525dd None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:12.770664 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a34aa0-300e-4cd5-9edd-39f9860eb0c4 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:12.803288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6f15c-d09b-4601-b575-b1b2ea3d5eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:12.803985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6f15c-d09b-4601-b575-b1b2ea3d5eeb None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:12.804266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6f15c-d09b-4601-b575-b1b2ea3d5eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:12.804536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b6f15c-d09b-4601-b575-b1b2ea3d5eeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:12.826469 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:12.856050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d51dc-4e8a-40bb-bc3f-24f1e09525dd None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:12.870920 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6f15c-d09b-4601-b575-b1b2ea3d5eeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:12.872399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b6f15c-d09b-4601-b575-b1b2ea3d5eeb None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:12.884241 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:12.889911 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6214d-5126-41aa-a8e8-26569d339b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:12.890529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6214d-5126-41aa-a8e8-26569d339b22 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:12.890847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6214d-5126-41aa-a8e8-26569d339b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:12.891173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a6214d-5126-41aa-a8e8-26569d339b22 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:12.898003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6214d-5126-41aa-a8e8-26569d339b22 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:12.902923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a6214d-5126-41aa-a8e8-26569d339b22 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-73a6214d-5126-41aa-a8e8-26569d339b22 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:12.911714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:12.912610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 252/519] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:12.913525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6214d-5126-41aa-a8e8-26569d339b22 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:12.971260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 268/520] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:12.972156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6f15c-d09b-4601-b575-b1b2ea3d5eeb None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.017598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a6214d-5126-41aa-a8e8-26569d339b22 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:13.032110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:13.050501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d9037-f769-437b-8436-cd0e6beaca9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:13.051120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9037-f769-437b-8436-cd0e6beaca9f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:13.051398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9037-f769-437b-8436-cd0e6beaca9f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:13.051668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d9037-f769-437b-8436-cd0e6beaca9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:13.062140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d9037-f769-437b-8436-cd0e6beaca9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:13.064165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d9037-f769-437b-8436-cd0e6beaca9f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-956d9037-f769-437b-8436-cd0e6beaca9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:13.077794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:13.078797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:13.079837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:13.079837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:13.079837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:13.079837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 253/521] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:13.079837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d9037-f769-437b-8436-cd0e6beaca9f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.080113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b6f15c-d09b-4601-b575-b1b2ea3d5eeb None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:13.115336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076db31-88a3-4e77-80d1-428436fcc944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:13.115984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076db31-88a3-4e77-80d1-428436fcc944 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:13.116261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076db31-88a3-4e77-80d1-428436fcc944 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:13.116530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076db31-88a3-4e77-80d1-428436fcc944 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:13.122132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:13.122946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076db31-88a3-4e77-80d1-428436fcc944 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:13.124437 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076db31-88a3-4e77-80d1-428436fcc944 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-1076db31-88a3-4e77-80d1-428436fcc944 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:13.137217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:13.138161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:13.139213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:13.139213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:13.139213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:13.139213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:13.139213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:13.139213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:13.139213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:13.139213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:13.139213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 269/522] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:13.139213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076db31-88a3-4e77-80d1-428436fcc944 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.181762 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d9037-f769-437b-8436-cd0e6beaca9f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:13.213738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be44e9a-fe9b-4c14-bae7-cac90457d64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:13.214326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be44e9a-fe9b-4c14-bae7-cac90457d64b None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:13.214607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be44e9a-fe9b-4c14-bae7-cac90457d64b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:13.214877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be44e9a-fe9b-4c14-bae7-cac90457d64b None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:13.215518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be44e9a-fe9b-4c14-bae7-cac90457d64b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:13.216955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be44e9a-fe9b-4c14-bae7-cac90457d64b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:13.237868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076db31-88a3-4e77-80d1-428436fcc944 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:13.250130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:13.270119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602128ba-14aa-49d0-afa5-06286bbdb934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:13.270661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602128ba-14aa-49d0-afa5-06286bbdb934 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:13.270940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602128ba-14aa-49d0-afa5-06286bbdb934 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:13.271216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602128ba-14aa-49d0-afa5-06286bbdb934 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:13.271963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602128ba-14aa-49d0-afa5-06286bbdb934 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:13.272269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602128ba-14aa-49d0-afa5-06286bbdb934 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:13.273700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602128ba-14aa-49d0-afa5-06286bbdb934 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:13.290325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 270/523] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Sep 7 16:04:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:13.291103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602128ba-14aa-49d0-afa5-06286bbdb934 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.295384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 254/524] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:04:13 2022] POST /identity/v3/users => generated 247 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:04:13.308859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be44e9a-fe9b-4c14-bae7-cac90457d64b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.400403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602128ba-14aa-49d0-afa5-06286bbdb934 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:13.421034 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:13.426461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be44e9a-fe9b-4c14-bae7-cac90457d64b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:13.437796 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee3c5b-1803-4454-bec7-3c4b0278ae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:13.438479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee3c5b-1803-4454-bec7-3c4b0278ae15 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:13.438759 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee3c5b-1803-4454-bec7-3c4b0278ae15 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:13.439036 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee3c5b-1803-4454-bec7-3c4b0278ae15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:13.449937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee3c5b-1803-4454-bec7-3c4b0278ae15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:13.451412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee3c5b-1803-4454-bec7-3c4b0278ae15 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:13.466081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ff8c2-11a5-49b8-b927-28e31438de32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:13.466694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ff8c2-11a5-49b8-b927-28e31438de32 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:13.466972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ff8c2-11a5-49b8-b927-28e31438de32 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:13.467244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ff8c2-11a5-49b8-b927-28e31438de32 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-e7ee3c5b-1803-4454-bec7-3c4b0278ae15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:13.467505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:13.469121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:13.476300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:13.476300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:13.476300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:13.476481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ff8c2-11a5-49b8-b927-28e31438de32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:13.476481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ff8c2-11a5-49b8-b927-28e31438de32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:13.476654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 271/525] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:13.476723 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ff8c2-11a5-49b8-b927-28e31438de32 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:13.476788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee3c5b-1803-4454-bec7-3c4b0278ae15 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.483107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 255/526] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:13.488148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ff8c2-11a5-49b8-b927-28e31438de32 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.505612 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:13.588968 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee3c5b-1803-4454-bec7-3c4b0278ae15 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:13.599939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ff8c2-11a5-49b8-b927-28e31438de32 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:13.625014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21132a13-6ab4-4eac-822c-30b8d53b8843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:13.625906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21132a13-6ab4-4eac-822c-30b8d53b8843 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:13.626249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21132a13-6ab4-4eac-822c-30b8d53b8843 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:13.626586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21132a13-6ab4-4eac-822c-30b8d53b8843 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:13.627340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21132a13-6ab4-4eac-822c-30b8d53b8843 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:13.627707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21132a13-6ab4-4eac-822c-30b8d53b8843 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:13.629225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21132a13-6ab4-4eac-822c-30b8d53b8843 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:13.634927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c59eb-540b-4ba7-bba6-698ba43eeae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:13.635535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c59eb-540b-4ba7-bba6-698ba43eeae9 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:13.635841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c59eb-540b-4ba7-bba6-698ba43eeae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:13.636117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c59eb-540b-4ba7-bba6-698ba43eeae9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:13.647889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 272/527] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:13.648697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21132a13-6ab4-4eac-822c-30b8d53b8843 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.670909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:13.716744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c59eb-540b-4ba7-bba6-698ba43eeae9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:13.718194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c59eb-540b-4ba7-bba6-698ba43eeae9 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:13.737520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:13.756511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21132a13-6ab4-4eac-822c-30b8d53b8843 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:13.791953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755316d-14c9-4dec-81b5-cb4a210f9a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:13.792562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755316d-14c9-4dec-81b5-cb4a210f9a16 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:13.792849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755316d-14c9-4dec-81b5-cb4a210f9a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:13.793127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755316d-14c9-4dec-81b5-cb4a210f9a16 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:13.793767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755316d-14c9-4dec-81b5-cb4a210f9a16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:13.794073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755316d-14c9-4dec-81b5-cb4a210f9a16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:13.795443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755316d-14c9-4dec-81b5-cb4a210f9a16 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:13.809599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 273/528] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:13.810266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755316d-14c9-4dec-81b5-cb4a210f9a16 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.828189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 256/529] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:13.829063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c59eb-540b-4ba7-bba6-698ba43eeae9 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.888303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:13.924331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755316d-14c9-4dec-81b5-cb4a210f9a16 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:13.932173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c59eb-540b-4ba7-bba6-698ba43eeae9 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:13.960736 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00979df0-3a87-45f0-b0fa-3d292cee0e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:13.961386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00979df0-3a87-45f0-b0fa-3d292cee0e25 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:13.961661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00979df0-3a87-45f0-b0fa-3d292cee0e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:13.961925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00979df0-3a87-45f0-b0fa-3d292cee0e25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:13.969938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00979df0-3a87-45f0-b0fa-3d292cee0e25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:13.970731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8ca17-d83e-4718-8434-3b07ba90380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:13.971584 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00979df0-3a87-45f0-b0fa-3d292cee0e25 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:13.971699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8ca17-d83e-4718-8434-3b07ba90380a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:13.972307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8ca17-d83e-4718-8434-3b07ba90380a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:13.972735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff8ca17-d83e-4718-8434-3b07ba90380a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:13.973747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8ca17-d83e-4718-8434-3b07ba90380a None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=45061db68b1e40f59935cc9a46d2ff10) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:13.974306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8ca17-d83e-4718-8434-3b07ba90380a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:13.975880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff8ca17-d83e-4718-8434-3b07ba90380a None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:13.981799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 257/530] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:13 2022] GET /identity/v3/role_assignments?user.id=45061db68b1e40f59935cc9a46d2ff10&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:13.982754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8ca17-d83e-4718-8434-3b07ba90380a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-00979df0-3a87-45f0-b0fa-3d292cee0e25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:13.983241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:13.984082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:13.985502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:13.985502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:13.985502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:13.985502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:13.985502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:13.985502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:13.985502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:13.985502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:13.985502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 274/531] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:13.985996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00979df0-3a87-45f0-b0fa-3d292cee0e25 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:13.999854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:14.121678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00979df0-3a87-45f0-b0fa-3d292cee0e25 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:14.141140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff8ca17-d83e-4718-8434-3b07ba90380a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:14.167103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:14.169919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d1408-c835-457d-a14b-e560c3e7e45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:14.170932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d1408-c835-457d-a14b-e560c3e7e45c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:14.171684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d1408-c835-457d-a14b-e560c3e7e45c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:14.172821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d1408-c835-457d-a14b-e560c3e7e45c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:14.180669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325fec2-e46e-46bb-ba7b-0c6d506e1c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:14.182748 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325fec2-e46e-46bb-ba7b-0c6d506e1c9c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:14.182869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325fec2-e46e-46bb-ba7b-0c6d506e1c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:14.183101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325fec2-e46e-46bb-ba7b-0c6d506e1c9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:14.187270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d1408-c835-457d-a14b-e560c3e7e45c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:14.188957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6d1408-c835-457d-a14b-e560c3e7e45c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:14.190589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325fec2-e46e-46bb-ba7b-0c6d506e1c9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:14.192219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325fec2-e46e-46bb-ba7b-0c6d506e1c9c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-3325fec2-e46e-46bb-ba7b-0c6d506e1c9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:14.206142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:14.207222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:14.208306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:14.208306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:14.208306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:14.208306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:14.208306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:14.208306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:14.208306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:14.208306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:14.214423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 258/532] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-0f6d1408-c835-457d-a14b-e560c3e7e45c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:14.214745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:14.215797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:14.216848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:14.216848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:14.216848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:14.217515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325fec2-e46e-46bb-ba7b-0c6d506e1c9c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:14.218388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 275/533] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:14.219935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d1408-c835-457d-a14b-e560c3e7e45c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:14.295429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:14.342293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325fec2-e46e-46bb-ba7b-0c6d506e1c9c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:14.342950 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6d1408-c835-457d-a14b-e560c3e7e45c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:14.379575 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c029e3-6c13-433f-bd89-6ba5e1d79f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:14.380399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c029e3-6c13-433f-bd89-6ba5e1d79f6f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:14.380865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c029e3-6c13-433f-bd89-6ba5e1d79f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:14.381156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c029e3-6c13-433f-bd89-6ba5e1d79f6f None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:14.382105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c029e3-6c13-433f-bd89-6ba5e1d79f6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:14.382343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c029e3-6c13-433f-bd89-6ba5e1d79f6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:14.383895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c029e3-6c13-433f-bd89-6ba5e1d79f6f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:14.388141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ce3c3-4190-4d66-9680-fbc73721f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:14.388930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ce3c3-4190-4d66-9680-fbc73721f79a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:14.389355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ce3c3-4190-4d66-9680-fbc73721f79a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:14.389795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55ce3c3-4190-4d66-9680-fbc73721f79a None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:14.390636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ce3c3-4190-4d66-9680-fbc73721f79a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:14.391679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ce3c3-4190-4d66-9680-fbc73721f79a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:14.393854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55ce3c3-4190-4d66-9680-fbc73721f79a None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:14.398660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 276/534] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:14.404445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c029e3-6c13-433f-bd89-6ba5e1d79f6f None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:14.416922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 259/535] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:14.417920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ce3c3-4190-4d66-9680-fbc73721f79a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:14.474680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:14.532552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c029e3-6c13-433f-bd89-6ba5e1d79f6f None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:14.539089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55ce3c3-4190-4d66-9680-fbc73721f79a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:14.549127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:14.572817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5af27-d33d-4379-a4c3-52aaa1e12a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:14.573439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5af27-d33d-4379-a4c3-52aaa1e12a64 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:14.573714 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5af27-d33d-4379-a4c3-52aaa1e12a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:14.573991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be5af27-d33d-4379-a4c3-52aaa1e12a64 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:14.574633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5af27-d33d-4379-a4c3-52aaa1e12a64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:14.574934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5af27-d33d-4379-a4c3-52aaa1e12a64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:14.576358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be5af27-d33d-4379-a4c3-52aaa1e12a64 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:14.577377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c1d4f-60fc-488f-b7fc-4b359355f4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:14.578142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c1d4f-60fc-488f-b7fc-4b359355f4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:14.579114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c1d4f-60fc-488f-b7fc-4b359355f4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:14.579386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9c1d4f-60fc-488f-b7fc-4b359355f4f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:14.580149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c1d4f-60fc-488f-b7fc-4b359355f4f5 None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=0a5d7cf7534941cb857a4ab5867d978e) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:14.580458 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c1d4f-60fc-488f-b7fc-4b359355f4f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:14.581960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9c1d4f-60fc-488f-b7fc-4b359355f4f5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:14.589248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 260/536] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:14 2022] GET /identity/v3/role_assignments?user.id=0a5d7cf7534941cb857a4ab5867d978e&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:14.590249 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c1d4f-60fc-488f-b7fc-4b359355f4f5 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:14.591194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 277/537] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:14.603710 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5af27-d33d-4379-a4c3-52aaa1e12a64 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:14.790683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9c1d4f-60fc-488f-b7fc-4b359355f4f5 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:14.795233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be5af27-d33d-4379-a4c3-52aaa1e12a64 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:14.804117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:14.837903 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92143cb-9be5-4ecf-9b62-e414716d6b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:14.838755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92143cb-9be5-4ecf-9b62-e414716d6b02 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:14.839031 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92143cb-9be5-4ecf-9b62-e414716d6b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:14.839300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92143cb-9be5-4ecf-9b62-e414716d6b02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:14.840050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92143cb-9be5-4ecf-9b62-e414716d6b02 None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=9f5f8dfbb9c44e788f110a4f53425344) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:14.840355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92143cb-9be5-4ecf-9b62-e414716d6b02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:14.841971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92143cb-9be5-4ecf-9b62-e414716d6b02 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:14.847985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 261/538] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:14 2022] GET /identity/v3/role_assignments?user.id=9f5f8dfbb9c44e788f110a4f53425344&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:14.862270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ff9ca-612c-4aaa-93c0-cfe8042aa7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:14.863022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ff9ca-612c-4aaa-93c0-cfe8042aa7fd None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:14.863308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ff9ca-612c-4aaa-93c0-cfe8042aa7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:14.863585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ff9ca-612c-4aaa-93c0-cfe8042aa7fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:14.872403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ff9ca-612c-4aaa-93c0-cfe8042aa7fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:14.873898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2ff9ca-612c-4aaa-93c0-cfe8042aa7fd None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-4b2ff9ca-612c-4aaa-93c0-cfe8042aa7fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:14.893503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:14.894416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:14.895264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:14.895264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:14.895264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:14.895264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:14.895264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:14.895264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:14.895264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:14.895264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:14.900315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 278/539] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:14.908923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92143cb-9be5-4ecf-9b62-e414716d6b02 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:15.063429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92143cb-9be5-4ecf-9b62-e414716d6b02 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:15.111500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5996f76-46e9-4f7a-a98e-64acddeb00d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:15.112135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5996f76-46e9-4f7a-a98e-64acddeb00d7 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.112416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5996f76-46e9-4f7a-a98e-64acddeb00d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.112691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5996f76-46e9-4f7a-a98e-64acddeb00d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.113379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5996f76-46e9-4f7a-a98e-64acddeb00d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:15.113688 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5996f76-46e9-4f7a-a98e-64acddeb00d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:15.115135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5996f76-46e9-4f7a-a98e-64acddeb00d7 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:15.131841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 262/540] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:15.133973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ff9ca-612c-4aaa-93c0-cfe8042aa7fd None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:15.196689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:15.280056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ff9ca-612c-4aaa-93c0-cfe8042aa7fd None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:15.326561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dab22-b829-42f2-92df-c8b266182bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:15.327089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dab22-b829-42f2-92df-c8b266182bfe None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.327420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dab22-b829-42f2-92df-c8b266182bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.327696 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dab22-b829-42f2-92df-c8b266182bfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.328533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dab22-b829-42f2-92df-c8b266182bfe None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=b6a6b52462624e7f84f479144353b51e) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:15.328848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dab22-b829-42f2-92df-c8b266182bfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:15.330283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dab22-b829-42f2-92df-c8b266182bfe None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:15.340339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 279/541] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:15 2022] GET /identity/v3/role_assignments?user.id=b6a6b52462624e7f84f479144353b51e&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:15.762662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d163248-bcc1-4eb9-a5f4-99f9e7753cfd None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.763655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d163248-bcc1-4eb9-a5f4-99f9e7753cfd None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.764176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d163248-bcc1-4eb9-a5f4-99f9e7753cfd None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.765976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 263/542] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:15.774407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d8545-87ca-4085-b1d5-ab6a76fac827 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.774850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d8545-87ca-4085-b1d5-ab6a76fac827 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.775029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d8545-87ca-4085-b1d5-ab6a76fac827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.804829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-652d8545-87ca-4085-b1d5-ab6a76fac827 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:15.806308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-652d8545-87ca-4085-b1d5-ab6a76fac827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:15.895524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 280/543] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:16.288524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d163248-bcc1-4eb9-a5f4-99f9e7753cfd None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:16.327021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:16.417807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d163248-bcc1-4eb9-a5f4-99f9e7753cfd None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:16.452680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca41c2-feff-4774-9de0-ada9eb881a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:16.453834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca41c2-feff-4774-9de0-ada9eb881a48 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:16.454129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca41c2-feff-4774-9de0-ada9eb881a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:16.454402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca41c2-feff-4774-9de0-ada9eb881a48 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:16.468223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca41c2-feff-4774-9de0-ada9eb881a48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-ecca41c2-feff-4774-9de0-ada9eb881a48 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:16.471418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:16.472338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:16.473354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 264/544] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:16.480186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d8545-87ca-4085-b1d5-ab6a76fac827 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:16.535829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:16.618805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d8545-87ca-4085-b1d5-ab6a76fac827 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:16.654815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba18db-26e5-44c8-89bf-37686a7bc2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:16.655750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba18db-26e5-44c8-89bf-37686a7bc2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:16.656246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba18db-26e5-44c8-89bf-37686a7bc2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:16.656753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba18db-26e5-44c8-89bf-37686a7bc2a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:16.657616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba18db-26e5-44c8-89bf-37686a7bc2a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:16.658120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba18db-26e5-44c8-89bf-37686a7bc2a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:16.659688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba18db-26e5-44c8-89bf-37686a7bc2a4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:16.668359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 281/545] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:16.675873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca41c2-feff-4774-9de0-ada9eb881a48 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:16.718559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31469322-2c09-4e22-9c85-2c35b99ef7e9 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:16.719201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31469322-2c09-4e22-9c85-2c35b99ef7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:16.719671 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31469322-2c09-4e22-9c85-2c35b99ef7e9 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:16.721608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 282/546] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:04:16.726424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8738c-e0fd-4310-8c11-76f344356fa8 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:16.726947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8738c-e0fd-4310-8c11-76f344356fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:16.727422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8738c-e0fd-4310-8c11-76f344356fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:16.745724 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-45d8738c-e0fd-4310-8c11-76f344356fa8 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:16.747175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-45d8738c-e0fd-4310-8c11-76f344356fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:16.847543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:16.850864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 283/547] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:16.859949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca41c2-feff-4774-9de0-ada9eb881a48 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:16.896295 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4119290-74ce-45d6-9159-3c0d7b097ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:16.897244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4119290-74ce-45d6-9159-3c0d7b097ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:16.897780 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4119290-74ce-45d6-9159-3c0d7b097ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:16.898342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4119290-74ce-45d6-9159-3c0d7b097ed7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:16.909647 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4119290-74ce-45d6-9159-3c0d7b097ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:16.911976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4119290-74ce-45d6-9159-3c0d7b097ed7 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-f4119290-74ce-45d6-9159-3c0d7b097ed7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:16.927976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:16.928898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:16.930245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:16.930245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:16.930245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:16.932635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 265/548] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:16.941177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8738c-e0fd-4310-8c11-76f344356fa8 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:16.973970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:17.073076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f58ca4-5dce-4dda-8146-d4c48facab2c None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:17.073407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f58ca4-5dce-4dda-8146-d4c48facab2c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:17.073626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f58ca4-5dce-4dda-8146-d4c48facab2c None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:17.074971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 266/549] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:04:17.080505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfec36-8e4c-4359-a90e-e9e33084a785 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:17.083658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfec36-8e4c-4359-a90e-e9e33084a785 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:17.083658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfec36-8e4c-4359-a90e-e9e33084a785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:17.106873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-61cfec36-8e4c-4359-a90e-e9e33084a785 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:17.108088 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-61cfec36-8e4c-4359-a90e-e9e33084a785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:17.124477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8738c-e0fd-4310-8c11-76f344356fa8 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:17.190893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60928646-4d77-4f0f-8a1d-cc41cef41807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:17.192206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928646-4d77-4f0f-8a1d-cc41cef41807 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:17.192688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928646-4d77-4f0f-8a1d-cc41cef41807 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:17.193635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60928646-4d77-4f0f-8a1d-cc41cef41807 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:17.194845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60928646-4d77-4f0f-8a1d-cc41cef41807 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:17.195390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60928646-4d77-4f0f-8a1d-cc41cef41807 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:17.198202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60928646-4d77-4f0f-8a1d-cc41cef41807 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:17.224035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 284/550] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:17.230921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60928646-4d77-4f0f-8a1d-cc41cef41807 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:17.236175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 267/551] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:17.311516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cfec36-8e4c-4359-a90e-e9e33084a785 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:17.373902 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:17.386991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:17.419518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cfec36-8e4c-4359-a90e-e9e33084a785 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:17.421134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60928646-4d77-4f0f-8a1d-cc41cef41807 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:17.463508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d1657-a895-4fd3-b180-009c9b39a0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:17.464096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d1657-a895-4fd3-b180-009c9b39a0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:17.464370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d1657-a895-4fd3-b180-009c9b39a0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:17.464637 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d1657-a895-4fd3-b180-009c9b39a0c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:17.482484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382d1657-a895-4fd3-b180-009c9b39a0c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-382d1657-a895-4fd3-b180-009c9b39a0c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:17.490221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:17.491141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:17.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:17.492824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 268/552] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:17.500573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d1657-a895-4fd3-b180-009c9b39a0c0 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:17.503547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154d07a-1847-41ba-9d9e-c7be9906f41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:17.504351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154d07a-1847-41ba-9d9e-c7be9906f41b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:17.504631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154d07a-1847-41ba-9d9e-c7be9906f41b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:17.504904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154d07a-1847-41ba-9d9e-c7be9906f41b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:17.523269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154d07a-1847-41ba-9d9e-c7be9906f41b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:17.525129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154d07a-1847-41ba-9d9e-c7be9906f41b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-6154d07a-1847-41ba-9d9e-c7be9906f41b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:17.541022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:17.542025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:17.542979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:17.542979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:17.542979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:17.542979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:17.542979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:17.542979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:17.542979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:17.542979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:17.542979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:17.542979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:17.542979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 285/553] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:17.543551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154d07a-1847-41ba-9d9e-c7be9906f41b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:17.643477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d1657-a895-4fd3-b180-009c9b39a0c0 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:17.680510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154d07a-1847-41ba-9d9e-c7be9906f41b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:17.683587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:17.689393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fa6b3-1a6b-4e23-931b-b1466f604e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:17.690279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fa6b3-1a6b-4e23-931b-b1466f604e71 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:17.690899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fa6b3-1a6b-4e23-931b-b1466f604e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:17.691163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488fa6b3-1a6b-4e23-931b-b1466f604e71 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:17.696610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fa6b3-1a6b-4e23-931b-b1466f604e71 None admin] RBAC: Adding query filter params (name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:17.696610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fa6b3-1a6b-4e23-931b-b1466f604e71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:17.696610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488fa6b3-1a6b-4e23-931b-b1466f604e71 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:17.711795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 269/554] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:17.711795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fa6b3-1a6b-4e23-931b-b1466f604e71 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:17.729963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470e01e-69d3-42b6-acd9-79a27052e49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:17.730612 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470e01e-69d3-42b6-acd9-79a27052e49f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:17.730958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470e01e-69d3-42b6-acd9-79a27052e49f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:17.731303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2470e01e-69d3-42b6-acd9-79a27052e49f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:17.740276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:17.743701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2470e01e-69d3-42b6-acd9-79a27052e49f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-2470e01e-69d3-42b6-acd9-79a27052e49f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:17.751056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:17.754440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:17.755410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 286/555] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:17.757641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470e01e-69d3-42b6-acd9-79a27052e49f None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:17.819836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488fa6b3-1a6b-4e23-931b-b1466f604e71 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:17.857164 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7102d-4148-4d40-93bf-4d2570abecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:17.857822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7102d-4148-4d40-93bf-4d2570abecd3 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:17.858326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7102d-4148-4d40-93bf-4d2570abecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:17.858601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7102d-4148-4d40-93bf-4d2570abecd3 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:17.859511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7102d-4148-4d40-93bf-4d2570abecd3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:17.859887 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7102d-4148-4d40-93bf-4d2570abecd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:17.861642 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7102d-4148-4d40-93bf-4d2570abecd3 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:17.868485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2470e01e-69d3-42b6-acd9-79a27052e49f None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:17.886342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 270/556] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 7 16:04:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:17.886342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09398cba-111e-4fa6-b4dc-bbf1912934bf None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:17.886342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09398cba-111e-4fa6-b4dc-bbf1912934bf None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:17.886342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09398cba-111e-4fa6-b4dc-bbf1912934bf None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:17.886342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 271/557] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:17.886342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09398cba-111e-4fa6-b4dc-bbf1912934bf None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:17.915140 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb37d7f-4a58-4d33-ae32-1a056a962b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:17.915792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb37d7f-4a58-4d33-ae32-1a056a962b09 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:17.916091 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb37d7f-4a58-4d33-ae32-1a056a962b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:17.916368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb37d7f-4a58-4d33-ae32-1a056a962b09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:17.930040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb37d7f-4a58-4d33-ae32-1a056a962b09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:17.931795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb37d7f-4a58-4d33-ae32-1a056a962b09 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-3cb37d7f-4a58-4d33-ae32-1a056a962b09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:17.946884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:17.948768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:17.951897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:17.951897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:17.951897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:17.951897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 287/558] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:17.951897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb37d7f-4a58-4d33-ae32-1a056a962b09 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:17.954669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:18.014010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09398cba-111e-4fa6-b4dc-bbf1912934bf None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:18.016692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:18.050049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e298a-7460-4f29-a456-4fe70009a682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:18.050749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e298a-7460-4f29-a456-4fe70009a682 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.051032 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e298a-7460-4f29-a456-4fe70009a682 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.051310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e298a-7460-4f29-a456-4fe70009a682 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.051996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e298a-7460-4f29-a456-4fe70009a682 None admin] RBAC: Adding query filter params (name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:18.052298 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e298a-7460-4f29-a456-4fe70009a682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:18.053655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e298a-7460-4f29-a456-4fe70009a682 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:18.067333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 272/559] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:18.067333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6df04-5cc6-4653-91d2-b08c7f355019 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.067333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6df04-5cc6-4653-91d2-b08c7f355019 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.067333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6df04-5cc6-4653-91d2-b08c7f355019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.072882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb37d7f-4a58-4d33-ae32-1a056a962b09 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:18.083334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-71e6df04-5cc6-4653-91d2-b08c7f355019 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:18.084568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-71e6df04-5cc6-4653-91d2-b08c7f355019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:18.106948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e102-6539-4199-a82e-eeb45f6b585d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:18.107563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e102-6539-4199-a82e-eeb45f6b585d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.107867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e102-6539-4199-a82e-eeb45f6b585d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.108139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900e102-6539-4199-a82e-eeb45f6b585d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.149706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 273/560] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:18.150520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6df04-5cc6-4653-91d2-b08c7f355019 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:18.182030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900e102-6539-4199-a82e-eeb45f6b585d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:18.183472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900e102-6539-4199-a82e-eeb45f6b585d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:18.203931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:18.238122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:18.261124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6df04-5cc6-4653-91d2-b08c7f355019 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:18.291454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 288/561] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:18.292548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e102-6539-4199-a82e-eeb45f6b585d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:18.301151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeed82b-603a-449c-9b2f-e70d7e5f4e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:18.301761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeed82b-603a-449c-9b2f-e70d7e5f4e16 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.302037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeed82b-603a-449c-9b2f-e70d7e5f4e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.302706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeed82b-603a-449c-9b2f-e70d7e5f4e16 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.303362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeed82b-603a-449c-9b2f-e70d7e5f4e16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:18.304034 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeed82b-603a-449c-9b2f-e70d7e5f4e16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:18.305414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeed82b-603a-449c-9b2f-e70d7e5f4e16 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:18.321991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 274/562] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:18.323233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffcd7b-a4ed-4dc8-be93-40039b464e22 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.323519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffcd7b-a4ed-4dc8-be93-40039b464e22 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.323803 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffcd7b-a4ed-4dc8-be93-40039b464e22 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.325090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 275/563] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:18.325771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffcd7b-a4ed-4dc8-be93-40039b464e22 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:18.396934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900e102-6539-4199-a82e-eeb45f6b585d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:18.430463 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4cf98-34cd-416e-95dd-a8390750d4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:18.431337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4cf98-34cd-416e-95dd-a8390750d4cf None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.431615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4cf98-34cd-416e-95dd-a8390750d4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.431915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f4cf98-34cd-416e-95dd-a8390750d4cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.439345 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffcd7b-a4ed-4dc8-be93-40039b464e22 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:18.441652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:18.442175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4cf98-34cd-416e-95dd-a8390750d4cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:18.443611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f4cf98-34cd-416e-95dd-a8390750d4cf None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-63f4cf98-34cd-416e-95dd-a8390750d4cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:18.457760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:18.460732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:18.461627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:18.461627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:18.461627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:18.461627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 289/564] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:18.461627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4cf98-34cd-416e-95dd-a8390750d4cf None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:18.464086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:18.472282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d54716-289e-4c3b-af93-f31a2e2ac7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:18.472905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d54716-289e-4c3b-af93-f31a2e2ac7bd None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.473181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d54716-289e-4c3b-af93-f31a2e2ac7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.473449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d54716-289e-4c3b-af93-f31a2e2ac7bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.483131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d54716-289e-4c3b-af93-f31a2e2ac7bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:18.485158 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d54716-289e-4c3b-af93-f31a2e2ac7bd None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-46d54716-289e-4c3b-af93-f31a2e2ac7bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:18.501093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:18.502209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:18.503102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:18.503102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:18.503102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:18.503102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 276/565] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:18.503335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2265-27bb-404e-981d-fc23e188dd13 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.503728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2265-27bb-404e-981d-fc23e188dd13 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.503916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a2265-27bb-404e-981d-fc23e188dd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.521373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c73a2265-27bb-404e-981d-fc23e188dd13 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:18.522496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c73a2265-27bb-404e-981d-fc23e188dd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:18.566577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f4cf98-34cd-416e-95dd-a8390750d4cf None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:18.590651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 277/566] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:18.591477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a2265-27bb-404e-981d-fc23e188dd13 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:18.599298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11c5f9-a2f1-4f9e-ba3f-e3cc79ae1789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:18.599965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11c5f9-a2f1-4f9e-ba3f-e3cc79ae1789 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.600247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11c5f9-a2f1-4f9e-ba3f-e3cc79ae1789 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.600522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11c5f9-a2f1-4f9e-ba3f-e3cc79ae1789 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.608087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11c5f9-a2f1-4f9e-ba3f-e3cc79ae1789 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:18.610366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d11c5f9-a2f1-4f9e-ba3f-e3cc79ae1789 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-4d11c5f9-a2f1-4f9e-ba3f-e3cc79ae1789 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:18.621622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:18.622699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 290/567] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 7 16:04:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:18.623653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11c5f9-a2f1-4f9e-ba3f-e3cc79ae1789 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:18.689651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:18.694676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:18.705449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a2265-27bb-404e-981d-fc23e188dd13 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:18.732845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11c5f9-a2f1-4f9e-ba3f-e3cc79ae1789 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:18.739585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f14967-a6ec-4ff4-8324-8c325af03c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:18.740510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f14967-a6ec-4ff4-8324-8c325af03c37 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.740996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f14967-a6ec-4ff4-8324-8c325af03c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.741566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f14967-a6ec-4ff4-8324-8c325af03c37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.755658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f14967-a6ec-4ff4-8324-8c325af03c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-84f14967-a6ec-4ff4-8324-8c325af03c37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:18.760860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:18.761785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:18.762640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:18.765515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 278/568] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:18.766414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f14967-a6ec-4ff4-8324-8c325af03c37 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:18.769068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4f127-9487-4c7f-bc44-ff053130fc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:18.769917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4f127-9487-4c7f-bc44-ff053130fc38 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.770393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4f127-9487-4c7f-bc44-ff053130fc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.770929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f4f127-9487-4c7f-bc44-ff053130fc38 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.771811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4f127-9487-4c7f-bc44-ff053130fc38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:18.772594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4f127-9487-4c7f-bc44-ff053130fc38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:18.773986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f4f127-9487-4c7f-bc44-ff053130fc38 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:18.786638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 291/569] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:18.787414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4f127-9487-4c7f-bc44-ff053130fc38 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:18.876531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f14967-a6ec-4ff4-8324-8c325af03c37 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:18.885657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f4f127-9487-4c7f-bc44-ff053130fc38 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:18.909448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:18.909716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078e275-9ddc-4ad7-9d8e-bd990521ee07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:18.910318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078e275-9ddc-4ad7-9d8e-bd990521ee07 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.910599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078e275-9ddc-4ad7-9d8e-bd990521ee07 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.910869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078e275-9ddc-4ad7-9d8e-bd990521ee07 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.911498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078e275-9ddc-4ad7-9d8e-bd990521ee07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:18.911823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078e275-9ddc-4ad7-9d8e-bd990521ee07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:18.913224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e078e275-9ddc-4ad7-9d8e-bd990521ee07 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:18.919810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31abcea-2315-4432-925a-4cc9a9cb3419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:18.920458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31abcea-2315-4432-925a-4cc9a9cb3419 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.920740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31abcea-2315-4432-925a-4cc9a9cb3419 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.921014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31abcea-2315-4432-925a-4cc9a9cb3419 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.921738 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31abcea-2315-4432-925a-4cc9a9cb3419 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:18.922694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31abcea-2315-4432-925a-4cc9a9cb3419 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:18.924138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31abcea-2315-4432-925a-4cc9a9cb3419 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:18.927072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 279/570] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:18.928155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078e275-9ddc-4ad7-9d8e-bd990521ee07 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:18.936831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:18.942572 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 292/571] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 7 16:04:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:18.943393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31abcea-2315-4432-925a-4cc9a9cb3419 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.037148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078e275-9ddc-4ad7-9d8e-bd990521ee07 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.045211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31abcea-2315-4432-925a-4cc9a9cb3419 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.070134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb04b22-f0b0-4761-92d8-2a174328c4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.070746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04b22-f0b0-4761-92d8-2a174328c4ac None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.071033 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04b22-f0b0-4761-92d8-2a174328c4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.071302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb04b22-f0b0-4761-92d8-2a174328c4ac None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.071999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb04b22-f0b0-4761-92d8-2a174328c4ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:19.072302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb04b22-f0b0-4761-92d8-2a174328c4ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:19.073707 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb04b22-f0b0-4761-92d8-2a174328c4ac None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:19.078486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2628c3-a492-41c0-b75a-8bfb8d788df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.079103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2628c3-a492-41c0-b75a-8bfb8d788df3 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.079380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2628c3-a492-41c0-b75a-8bfb8d788df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.079651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2628c3-a492-41c0-b75a-8bfb8d788df3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.081698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 280/572] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:19.082511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb04b22-f0b0-4761-92d8-2a174328c4ac None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.086812 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2628c3-a492-41c0-b75a-8bfb8d788df3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:19.088299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2628c3-a492-41c0-b75a-8bfb8d788df3 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-de2628c3-a492-41c0-b75a-8bfb8d788df3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:19.100184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:19.101361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 293/573] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:19.102331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2628c3-a492-41c0-b75a-8bfb8d788df3 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.143520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:19.154243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:19.187207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb04b22-f0b0-4761-92d8-2a174328c4ac None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.198919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2628c3-a492-41c0-b75a-8bfb8d788df3 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.220722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbd600-db4c-4246-aa1c-e0a664d44c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.221337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbd600-db4c-4246-aa1c-e0a664d44c23 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.221616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbd600-db4c-4246-aa1c-e0a664d44c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.221885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbd600-db4c-4246-aa1c-e0a664d44c23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.233833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbd600-db4c-4246-aa1c-e0a664d44c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:19.236699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b1cf7-27b0-41a8-8def-103e26fe9ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.237351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b1cf7-27b0-41a8-8def-103e26fe9ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.237614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b1cf7-27b0-41a8-8def-103e26fe9ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.237892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b1cf7-27b0-41a8-8def-103e26fe9ad2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-a0bbd600-db4c-4246-aa1c-e0a664d44c23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:19.239423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:19.241313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 281/574] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:19.242285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbd600-db4c-4246-aa1c-e0a664d44c23 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.250756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b1cf7-27b0-41a8-8def-103e26fe9ad2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:19.252398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b1cf7-27b0-41a8-8def-103e26fe9ad2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-ed1b1cf7-27b0-41a8-8def-103e26fe9ad2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:19.263745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:19.265009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:19.267034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:19.267034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:19.267034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:19.267034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:19.267034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:19.267034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:19.267034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:19.267034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:19.267034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 294/575] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:19.267034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b1cf7-27b0-41a8-8def-103e26fe9ad2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.340314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbd600-db4c-4246-aa1c-e0a664d44c23 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.365568 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b1cf7-27b0-41a8-8def-103e26fe9ad2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.366381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:19.367155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:19.373846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303b4e0-d5b9-4933-a2b9-4dcf7890044f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.374464 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303b4e0-d5b9-4933-a2b9-4dcf7890044f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.374741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303b4e0-d5b9-4933-a2b9-4dcf7890044f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.375045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303b4e0-d5b9-4933-a2b9-4dcf7890044f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.385241 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303b4e0-d5b9-4933-a2b9-4dcf7890044f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:19.386687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303b4e0-d5b9-4933-a2b9-4dcf7890044f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:19.398415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d5964-6e12-4d14-9aac-891cdcdd7306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.399031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d5964-6e12-4d14-9aac-891cdcdd7306 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.399308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d5964-6e12-4d14-9aac-891cdcdd7306 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.399577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d5964-6e12-4d14-9aac-891cdcdd7306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-1303b4e0-d5b9-4933-a2b9-4dcf7890044f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:19.401142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:19.402054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:19.402939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:19.402939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:19.402939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:19.403108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 282/576] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:19.403966 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303b4e0-d5b9-4933-a2b9-4dcf7890044f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.411981 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d5964-6e12-4d14-9aac-891cdcdd7306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:19.418295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d5964-6e12-4d14-9aac-891cdcdd7306 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-2c3d5964-6e12-4d14-9aac-891cdcdd7306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:19.427412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:19.428450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:19.429321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:19.429321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:19.429321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:19.429321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 295/577] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:19.429550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d5964-6e12-4d14-9aac-891cdcdd7306 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.504825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303b4e0-d5b9-4933-a2b9-4dcf7890044f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.529156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d5964-6e12-4d14-9aac-891cdcdd7306 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.538688 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977d620-1840-4b75-989e-62bb4718a092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.539283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977d620-1840-4b75-989e-62bb4718a092 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.539559 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977d620-1840-4b75-989e-62bb4718a092 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.539869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977d620-1840-4b75-989e-62bb4718a092 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.540582 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977d620-1840-4b75-989e-62bb4718a092 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:19.540881 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977d620-1840-4b75-989e-62bb4718a092 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:19.542295 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977d620-1840-4b75-989e-62bb4718a092 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:19.558856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 283/578] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:19.559625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977d620-1840-4b75-989e-62bb4718a092 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.561670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c6dd9-68f9-4d82-8494-3a700dcb16da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.562448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c6dd9-68f9-4d82-8494-3a700dcb16da None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.564003 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c6dd9-68f9-4d82-8494-3a700dcb16da None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.564293 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c6dd9-68f9-4d82-8494-3a700dcb16da None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.564961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c6dd9-68f9-4d82-8494-3a700dcb16da None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:19.565266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c6dd9-68f9-4d82-8494-3a700dcb16da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:19.566622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c6dd9-68f9-4d82-8494-3a700dcb16da None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:19.575161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 296/579] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:19.575970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c6dd9-68f9-4d82-8494-3a700dcb16da None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.596142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:19.600602 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:19.666701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977d620-1840-4b75-989e-62bb4718a092 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.676567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c6dd9-68f9-4d82-8494-3a700dcb16da None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.699928 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3e633-957b-4529-ba99-b17c46367958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.700524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3e633-957b-4529-ba99-b17c46367958 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.700801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3e633-957b-4529-ba99-b17c46367958 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.701074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3e633-957b-4529-ba99-b17c46367958 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.701758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3e633-957b-4529-ba99-b17c46367958 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:19.702062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3e633-957b-4529-ba99-b17c46367958 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:19.704114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3e633-957b-4529-ba99-b17c46367958 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:19.710005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cba7f3-4b98-46b5-8d09-182eef7307b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.710607 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba7f3-4b98-46b5-8d09-182eef7307b0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.710883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba7f3-4b98-46b5-8d09-182eef7307b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.711153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cba7f3-4b98-46b5-8d09-182eef7307b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.711795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cba7f3-4b98-46b5-8d09-182eef7307b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:19.712097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cba7f3-4b98-46b5-8d09-182eef7307b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:19.713440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cba7f3-4b98-46b5-8d09-182eef7307b0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:19.715255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 284/580] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:19.716066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3e633-957b-4529-ba99-b17c46367958 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.729623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 297/581] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:19.730461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cba7f3-4b98-46b5-8d09-182eef7307b0 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.805400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:19.815605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:19.819440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3e633-957b-4529-ba99-b17c46367958 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.827664 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cba7f3-4b98-46b5-8d09-182eef7307b0 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.851950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ae932-106a-4e83-84e4-73037b35bd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.852492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ae932-106a-4e83-84e4-73037b35bd3b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.852772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ae932-106a-4e83-84e4-73037b35bd3b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.853047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ae932-106a-4e83-84e4-73037b35bd3b None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.853682 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ae932-106a-4e83-84e4-73037b35bd3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:19.853983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ae932-106a-4e83-84e4-73037b35bd3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:19.855334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ae932-106a-4e83-84e4-73037b35bd3b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:19.860769 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb0f99-88d8-4330-8ca6-2f88a44f9eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:19.861386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb0f99-88d8-4330-8ca6-2f88a44f9eda None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:19.861672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb0f99-88d8-4330-8ca6-2f88a44f9eda None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:19.861946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adb0f99-88d8-4330-8ca6-2f88a44f9eda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:19.866710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 285/582] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:19.867487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ae932-106a-4e83-84e4-73037b35bd3b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.872835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb0f99-88d8-4330-8ca6-2f88a44f9eda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:19.874293 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adb0f99-88d8-4330-8ca6-2f88a44f9eda None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-5adb0f99-88d8-4330-8ca6-2f88a44f9eda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:19.889180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:19.897224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:19.898653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:19.898653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:19.898653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:19.898653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 298/583] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:19.898653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb0f99-88d8-4330-8ca6-2f88a44f9eda None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:19.975711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ae932-106a-4e83-84e4-73037b35bd3b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:19.994642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adb0f99-88d8-4330-8ca6-2f88a44f9eda None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.010244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f373cd7-866b-44bf-b3d7-9f576a7e40e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.010882 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f373cd7-866b-44bf-b3d7-9f576a7e40e0 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.011168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f373cd7-866b-44bf-b3d7-9f576a7e40e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.011446 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f373cd7-866b-44bf-b3d7-9f576a7e40e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.022334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f373cd7-866b-44bf-b3d7-9f576a7e40e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.022895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:20.023791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f373cd7-866b-44bf-b3d7-9f576a7e40e0 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.030329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90805a57-631b-45af-b2c8-e3329ded1946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.030926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90805a57-631b-45af-b2c8-e3329ded1946 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.031229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90805a57-631b-45af-b2c8-e3329ded1946 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.031505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90805a57-631b-45af-b2c8-e3329ded1946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.032256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90805a57-631b-45af-b2c8-e3329ded1946 None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=69a8796195ff4dfb85655cdd91a0229d) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:20.035565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90805a57-631b-45af-b2c8-e3329ded1946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.035565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90805a57-631b-45af-b2c8-e3329ded1946 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-5f373cd7-866b-44bf-b3d7-9f576a7e40e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:20.037812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:20.038879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:20.040081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:20.040081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:20.040081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:20.040081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 286/584] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:20.040081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f373cd7-866b-44bf-b3d7-9f576a7e40e0 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.044073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 299/585] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:19 2022] GET /identity/v3/role_assignments?user.id=69a8796195ff4dfb85655cdd91a0229d&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:20.044953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90805a57-631b-45af-b2c8-e3329ded1946 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.058936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:20.157173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90805a57-631b-45af-b2c8-e3329ded1946 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.160317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f373cd7-866b-44bf-b3d7-9f576a7e40e0 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.201450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c29e13-24a8-41e6-8f62-bd39c44e3e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.202189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29e13-24a8-41e6-8f62-bd39c44e3e86 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.202592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29e13-24a8-41e6-8f62-bd39c44e3e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.202963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c29e13-24a8-41e6-8f62-bd39c44e3e86 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.208707 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e26cbe-1fc0-4084-b033-f3dfcccd751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.209656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e26cbe-1fc0-4084-b033-f3dfcccd751b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.210072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e26cbe-1fc0-4084-b033-f3dfcccd751b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.210489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e26cbe-1fc0-4084-b033-f3dfcccd751b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.214984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c29e13-24a8-41e6-8f62-bd39c44e3e86 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.216965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c29e13-24a8-41e6-8f62-bd39c44e3e86 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.220783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e26cbe-1fc0-4084-b033-f3dfcccd751b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.222213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e26cbe-1fc0-4084-b033-f3dfcccd751b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-54e26cbe-1fc0-4084-b033-f3dfcccd751b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:20.233755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:20.236133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:20.237808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:20.237808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:20.237808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:20.237808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:20.237808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:20.237808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:20.237808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:20.237808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-56c29e13-24a8-41e6-8f62-bd39c44e3e86 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:20.238733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:20.241903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 300/586] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:20.246146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c29e13-24a8-41e6-8f62-bd39c44e3e86 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.248471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 287/587] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:20.248471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e26cbe-1fc0-4084-b033-f3dfcccd751b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.298765 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:20.317793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:20.348106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c29e13-24a8-41e6-8f62-bd39c44e3e86 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.351018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e26cbe-1fc0-4084-b033-f3dfcccd751b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.383529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac2f7c-91a3-4ee7-a43d-0bf91aa47635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.384172 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac2f7c-91a3-4ee7-a43d-0bf91aa47635 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.384452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac2f7c-91a3-4ee7-a43d-0bf91aa47635 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.384727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac2f7c-91a3-4ee7-a43d-0bf91aa47635 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.385362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac2f7c-91a3-4ee7-a43d-0bf91aa47635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:20.385663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac2f7c-91a3-4ee7-a43d-0bf91aa47635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.387030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac2f7c-91a3-4ee7-a43d-0bf91aa47635 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.387930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba687dac-13bb-4d4f-8223-8ac94f8af341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.388630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba687dac-13bb-4d4f-8223-8ac94f8af341 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.389005 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba687dac-13bb-4d4f-8223-8ac94f8af341 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.389660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba687dac-13bb-4d4f-8223-8ac94f8af341 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.390059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba687dac-13bb-4d4f-8223-8ac94f8af341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:20.390424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba687dac-13bb-4d4f-8223-8ac94f8af341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.391888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba687dac-13bb-4d4f-8223-8ac94f8af341 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.400131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 301/588] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:20.400827 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac2f7c-91a3-4ee7-a43d-0bf91aa47635 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.405009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 288/589] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:20.405434 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba687dac-13bb-4d4f-8223-8ac94f8af341 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.510027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba687dac-13bb-4d4f-8223-8ac94f8af341 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.513994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac2f7c-91a3-4ee7-a43d-0bf91aa47635 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.533594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:20.542510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:20.549880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3f3b1-9c24-43b5-a88c-fe77016298ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.550675 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3f3b1-9c24-43b5-a88c-fe77016298ec None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.551102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3f3b1-9c24-43b5-a88c-fe77016298ec None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.551519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc3f3b1-9c24-43b5-a88c-fe77016298ec None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.560124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3f3b1-9c24-43b5-a88c-fe77016298ec None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:20.560124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3f3b1-9c24-43b5-a88c-fe77016298ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.560124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc3f3b1-9c24-43b5-a88c-fe77016298ec None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.561130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb303b9-a8b5-4088-8917-7fc99c8b7f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.561758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb303b9-a8b5-4088-8917-7fc99c8b7f0d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.562043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb303b9-a8b5-4088-8917-7fc99c8b7f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.562316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb303b9-a8b5-4088-8917-7fc99c8b7f0d None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.563337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb303b9-a8b5-4088-8917-7fc99c8b7f0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:20.563859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb303b9-a8b5-4088-8917-7fc99c8b7f0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.565729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb303b9-a8b5-4088-8917-7fc99c8b7f0d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.575103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 289/590] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 7 16:04:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:20.576228 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3f3b1-9c24-43b5-a88c-fe77016298ec None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.585894 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 302/591] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:20.585894 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb303b9-a8b5-4088-8917-7fc99c8b7f0d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.687537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc3f3b1-9c24-43b5-a88c-fe77016298ec None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.693750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb303b9-a8b5-4088-8917-7fc99c8b7f0d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.724964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d3f07-48b1-4dd1-a42c-e21437353706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.725746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d3f07-48b1-4dd1-a42c-e21437353706 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.726118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d3f07-48b1-4dd1-a42c-e21437353706 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.726487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d3f07-48b1-4dd1-a42c-e21437353706 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.729348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8d307-6784-4108-9da0-0830715d2b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.729964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8d307-6784-4108-9da0-0830715d2b9b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.730246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8d307-6784-4108-9da0-0830715d2b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.730521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d8d307-6784-4108-9da0-0830715d2b9b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.734298 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d3f07-48b1-4dd1-a42c-e21437353706 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.735902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d3f07-48b1-4dd1-a42c-e21437353706 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.738866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8d307-6784-4108-9da0-0830715d2b9b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.740328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d8d307-6784-4108-9da0-0830715d2b9b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-162d3f07-48b1-4dd1-a42c-e21437353706 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:20.747283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:20.748367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:20.749436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:20.749436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:20.749436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:20.749436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:20.749436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:20.749436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:20.749436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:20.749436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:20.750874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 290/592] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-c2d8d307-6784-4108-9da0-0830715d2b9b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:20.751709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:20.752649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:20.753588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:20.753588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:20.753588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:20.753588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:20.753588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:20.753588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:20.753588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:20.753588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:20.753588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 16:04:20.753588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:20.754279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d3f07-48b1-4dd1-a42c-e21437353706 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.757641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 303/593] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:20.758502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8d307-6784-4108-9da0-0830715d2b9b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.791709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:20.806563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:20.860074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d3f07-48b1-4dd1-a42c-e21437353706 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.868572 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d8d307-6784-4108-9da0-0830715d2b9b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.895786 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f93e05-de4f-4e4a-bd9b-b3d42bf10fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.896420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f93e05-de4f-4e4a-bd9b-b3d42bf10fca None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.896697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f93e05-de4f-4e4a-bd9b-b3d42bf10fca None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.896970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f93e05-de4f-4e4a-bd9b-b3d42bf10fca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.908229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f93e05-de4f-4e4a-bd9b-b3d42bf10fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.909686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f93e05-de4f-4e4a-bd9b-b3d42bf10fca None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.913499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac2df4-5cb0-4cb1-8f63-ada60ad45705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.914654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac2df4-5cb0-4cb1-8f63-ada60ad45705 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.915096 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac2df4-5cb0-4cb1-8f63-ada60ad45705 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.915511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac2df4-5cb0-4cb1-8f63-ada60ad45705 None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/45061db68b1e40f59935cc9a46d2ff10/roles/d9477edba6304c829c6348bc4e6e6c1d` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-a0f93e05-de4f-4e4a-bd9b-b3d42bf10fca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:20.924377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:20.925585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:20.926728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:20.926728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:20.926728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:20.930394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 291/594] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:20.931632 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f93e05-de4f-4e4a-bd9b-b3d42bf10fca None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.937849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac2df4-5cb0-4cb1-8f63-ada60ad45705 None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=45061db68b1e40f59935cc9a46d2ff10, role_id=d9477edba6304c829c6348bc4e6e6c1d)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.939288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac2df4-5cb0-4cb1-8f63-ada60ad45705 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:21.035798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 304/595] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:04:20 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/45061db68b1e40f59935cc9a46d2ff10/roles/d9477edba6304c829c6348bc4e6e6c1d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:21.035798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac2df4-5cb0-4cb1-8f63-ada60ad45705 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:21.049264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:21.052457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f93e05-de4f-4e4a-bd9b-b3d42bf10fca None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:21.102050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c630dcc-b8d6-4812-89df-161edf979d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:21.103201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c630dcc-b8d6-4812-89df-161edf979d4d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:21.103498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c630dcc-b8d6-4812-89df-161edf979d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:21.103794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c630dcc-b8d6-4812-89df-161edf979d4d None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:21.104835 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c630dcc-b8d6-4812-89df-161edf979d4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:21.105146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c630dcc-b8d6-4812-89df-161edf979d4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:21.106767 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c630dcc-b8d6-4812-89df-161edf979d4d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:21.130717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 292/596] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:21.131725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c630dcc-b8d6-4812-89df-161edf979d4d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:21.138577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:21.159914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac2df4-5cb0-4cb1-8f63-ada60ad45705 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:21.200517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1b0a6-7ac3-4f02-9dd3-c62e096ea297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:21.202470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b0a6-7ac3-4f02-9dd3-c62e096ea297 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:21.202907 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b0a6-7ac3-4f02-9dd3-c62e096ea297 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:21.203923 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1b0a6-7ac3-4f02-9dd3-c62e096ea297 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:21.204813 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1b0a6-7ac3-4f02-9dd3-c62e096ea297 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:21.205317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1b0a6-7ac3-4f02-9dd3-c62e096ea297 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:21.207017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b1b0a6-7ac3-4f02-9dd3-c62e096ea297 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:21.226860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 305/597] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Sep 7 16:04:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:21.227962 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1b0a6-7ac3-4f02-9dd3-c62e096ea297 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:21.260754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c630dcc-b8d6-4812-89df-161edf979d4d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:21.296623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44c6fd-2d66-4980-a3dc-930c70089ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:21.297616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44c6fd-2d66-4980-a3dc-930c70089ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:21.298044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44c6fd-2d66-4980-a3dc-930c70089ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:21.298475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44c6fd-2d66-4980-a3dc-930c70089ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:21.299119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44c6fd-2d66-4980-a3dc-930c70089ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:21.299478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44c6fd-2d66-4980-a3dc-930c70089ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:21.300821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44c6fd-2d66-4980-a3dc-930c70089ca4 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:21.313226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 293/598] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:21.314041 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44c6fd-2d66-4980-a3dc-930c70089ca4 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:21.336608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:21.350284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1b0a6-7ac3-4f02-9dd3-c62e096ea297 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:21.395342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefd049-ac79-475a-80c8-800d7c8d5215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:21.397497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefd049-ac79-475a-80c8-800d7c8d5215 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:21.397990 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefd049-ac79-475a-80c8-800d7c8d5215 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:21.398462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefd049-ac79-475a-80c8-800d7c8d5215 None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/0a5d7cf7534941cb857a4ab5867d978e/roles/d9477edba6304c829c6348bc4e6e6c1d` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:21.412889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:21.417555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefd049-ac79-475a-80c8-800d7c8d5215 None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=0a5d7cf7534941cb857a4ab5867d978e, role_id=d9477edba6304c829c6348bc4e6e6c1d)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:21.419824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefd049-ac79-475a-80c8-800d7c8d5215 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:21.433426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44c6fd-2d66-4980-a3dc-930c70089ca4 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:21.468655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a29997-edc6-4502-9f22-1a3b28f252bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:21.469283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a29997-edc6-4502-9f22-1a3b28f252bf None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:21.469564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a29997-edc6-4502-9f22-1a3b28f252bf None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:21.469837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a29997-edc6-4502-9f22-1a3b28f252bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:21.480534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a29997-edc6-4502-9f22-1a3b28f252bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:21.481967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a29997-edc6-4502-9f22-1a3b28f252bf None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-b5a29997-edc6-4502-9f22-1a3b28f252bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:21.496932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:21.497969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:21.498872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:21.498872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:21.498872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:21.498872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 294/599] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:21.499606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a29997-edc6-4502-9f22-1a3b28f252bf None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:21.507791 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 306/600] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:04:21 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/0a5d7cf7534941cb857a4ab5867d978e/roles/d9477edba6304c829c6348bc4e6e6c1d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:21.507791 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefd049-ac79-475a-80c8-800d7c8d5215 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:21.643313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:21.679224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a29997-edc6-4502-9f22-1a3b28f252bf None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:21.728120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dbd9c-ea2d-47eb-98f3-91a6d96d466f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:21.728857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dbd9c-ea2d-47eb-98f3-91a6d96d466f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:21.729219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dbd9c-ea2d-47eb-98f3-91a6d96d466f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:21.729557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6dbd9c-ea2d-47eb-98f3-91a6d96d466f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:21.739230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6dbd9c-ea2d-47eb-98f3-91a6d96d466f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:21.740691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6dbd9c-ea2d-47eb-98f3-91a6d96d466f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:21.746099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefd049-ac79-475a-80c8-800d7c8d5215 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-1a6dbd9c-ea2d-47eb-98f3-91a6d96d466f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:21.755955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:21.757078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:21.758150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:21.758150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:21.758150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:21.758150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:21.758150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:21.758150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:21.758150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:21.758150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:21.759837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 295/601] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:21.765838 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dbd9c-ea2d-47eb-98f3-91a6d96d466f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:21.792188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf66ba9-e5b2-47ca-9321-729eda5d9044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:21.793325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf66ba9-e5b2-47ca-9321-729eda5d9044 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:21.793752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf66ba9-e5b2-47ca-9321-729eda5d9044 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:21.794190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf66ba9-e5b2-47ca-9321-729eda5d9044 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:21.795225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf66ba9-e5b2-47ca-9321-729eda5d9044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:21.796270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf66ba9-e5b2-47ca-9321-729eda5d9044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:21.798964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf66ba9-e5b2-47ca-9321-729eda5d9044 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:21.816468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 307/602] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:21.822361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf66ba9-e5b2-47ca-9321-729eda5d9044 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:21.881628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6dbd9c-ea2d-47eb-98f3-91a6d96d466f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:21.894678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:21.920142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:21.926292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95383947-7beb-4c94-ac78-97874348ac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:21.927347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95383947-7beb-4c94-ac78-97874348ac1d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:21.927347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95383947-7beb-4c94-ac78-97874348ac1d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:21.927781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95383947-7beb-4c94-ac78-97874348ac1d None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:21.928587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95383947-7beb-4c94-ac78-97874348ac1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:21.929000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95383947-7beb-4c94-ac78-97874348ac1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:21.930624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95383947-7beb-4c94-ac78-97874348ac1d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:21.946282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 296/603] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:21.952155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95383947-7beb-4c94-ac78-97874348ac1d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:21.966093 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf66ba9-e5b2-47ca-9321-729eda5d9044 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:22.036510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c9716-8a67-4eba-953a-28de847da124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:22.037156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9716-8a67-4eba-953a-28de847da124 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:22.037436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9716-8a67-4eba-953a-28de847da124 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:22.037713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721c9716-8a67-4eba-953a-28de847da124 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:22.056704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c9716-8a67-4eba-953a-28de847da124 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:22.058161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721c9716-8a67-4eba-953a-28de847da124 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:22.078170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95383947-7beb-4c94-ac78-97874348ac1d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-721c9716-8a67-4eba-953a-28de847da124 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:22.081444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:22.082321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:22.083213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:22.083213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:22.083213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:22.083213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:22.083213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:22.083213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:22.083213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:22.083213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:22.083213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 308/604] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:22.092782 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c9716-8a67-4eba-953a-28de847da124 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:22.117893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff502b-c52d-4959-80fe-07b547323f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:22.118639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff502b-c52d-4959-80fe-07b547323f1d None admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:22.119251 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff502b-c52d-4959-80fe-07b547323f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:22.119475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff502b-c52d-4959-80fe-07b547323f1d None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/9f5f8dfbb9c44e788f110a4f53425344/roles/d9477edba6304c829c6348bc4e6e6c1d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:22.135540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff502b-c52d-4959-80fe-07b547323f1d None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=9f5f8dfbb9c44e788f110a4f53425344, role_id=d9477edba6304c829c6348bc4e6e6c1d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:22.137199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff502b-c52d-4959-80fe-07b547323f1d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:22.226561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 297/605] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:04:21 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/9f5f8dfbb9c44e788f110a4f53425344/roles/d9477edba6304c829c6348bc4e6e6c1d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:22.240345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721c9716-8a67-4eba-953a-28de847da124 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:22.262351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:22.288469 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676e6f85-b1e7-401d-a2db-e98bc1ac8a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:22.289093 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e6f85-b1e7-401d-a2db-e98bc1ac8a62 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:22.289371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e6f85-b1e7-401d-a2db-e98bc1ac8a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:22.289646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e6f85-b1e7-401d-a2db-e98bc1ac8a62 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:22.290338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e6f85-b1e7-401d-a2db-e98bc1ac8a62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:22.290641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e6f85-b1e7-401d-a2db-e98bc1ac8a62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:22.292140 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676e6f85-b1e7-401d-a2db-e98bc1ac8a62 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:22.312380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 309/606] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:22.318619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff502b-c52d-4959-80fe-07b547323f1d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:22.372240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9bdd4-7788-4b2f-9507-e2431763dd53 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:22.372604 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9bdd4-7788-4b2f-9507-e2431763dd53 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:22.372931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9bdd4-7788-4b2f-9507-e2431763dd53 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:22.374296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 310/607] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:22.380153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951641f2-7cad-422b-931b-e29599cdcfe2 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:22.380656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951641f2-7cad-422b-931b-e29599cdcfe2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:22.381080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951641f2-7cad-422b-931b-e29599cdcfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:22.384862 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:22.412760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-951641f2-7cad-422b-931b-e29599cdcfe2 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:22.415560 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-951641f2-7cad-422b-931b-e29599cdcfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:22.494393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff502b-c52d-4959-80fe-07b547323f1d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:22.554377 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 311/608] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:22.567308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6224af1-e463-4400-997d-2c5da393d959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:22.568004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6224af1-e463-4400-997d-2c5da393d959 None admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:22.568279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6224af1-e463-4400-997d-2c5da393d959 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:22.568547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6224af1-e463-4400-997d-2c5da393d959 None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/b6a6b52462624e7f84f479144353b51e/roles/d9477edba6304c829c6348bc4e6e6c1d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:22.592703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6224af1-e463-4400-997d-2c5da393d959 None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=b6a6b52462624e7f84f479144353b51e, role_id=d9477edba6304c829c6348bc4e6e6c1d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:22.594188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6224af1-e463-4400-997d-2c5da393d959 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:22.665609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 298/609] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:04:22 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/b6a6b52462624e7f84f479144353b51e/roles/d9477edba6304c829c6348bc4e6e6c1d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:22.832944 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951641f2-7cad-422b-931b-e29599cdcfe2 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:22.960501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:22.963373 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951641f2-7cad-422b-931b-e29599cdcfe2 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:23.003032 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882731f-4cf5-495f-b0e5-6ee20578858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:23.003656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882731f-4cf5-495f-b0e5-6ee20578858b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:23.003969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882731f-4cf5-495f-b0e5-6ee20578858b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:23.004244 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882731f-4cf5-495f-b0e5-6ee20578858b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:23.021926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882731f-4cf5-495f-b0e5-6ee20578858b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-4882731f-4cf5-495f-b0e5-6ee20578858b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:23.027423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:23.028391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:23.029223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 312/610] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:23.036326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6224af1-e463-4400-997d-2c5da393d959 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:23.164674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:23.174042 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6224af1-e463-4400-997d-2c5da393d959 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:23.209948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4c3f-58fb-4bd6-b493-f9420c59a074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:23.210564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4c3f-58fb-4bd6-b493-f9420c59a074 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:23.210841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4c3f-58fb-4bd6-b493-f9420c59a074 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:23.211116 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df4c3f-58fb-4bd6-b493-f9420c59a074 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:23.211799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4c3f-58fb-4bd6-b493-f9420c59a074 None admin] RBAC: Adding query filter params (name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:23.212105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4c3f-58fb-4bd6-b493-f9420c59a074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:23.213473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df4c3f-58fb-4bd6-b493-f9420c59a074 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:23.222123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 299/611] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:23.227658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882731f-4cf5-495f-b0e5-6ee20578858b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:23.380677 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882731f-4cf5-495f-b0e5-6ee20578858b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:23.424469 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4ac4b-54e6-4658-bf17-b478549c372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:23.425021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4ac4b-54e6-4658-bf17-b478549c372d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:23.425307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4ac4b-54e6-4658-bf17-b478549c372d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:23.425580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4ac4b-54e6-4658-bf17-b478549c372d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:23.440725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4ac4b-54e6-4658-bf17-b478549c372d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:23.442191 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4ac4b-54e6-4658-bf17-b478549c372d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-3ef4ac4b-54e6-4658-bf17-b478549c372d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:23.459868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:23.460883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:23.461792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:23.461792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:23.461792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:23.461792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 313/612] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:23.468325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4c3f-58fb-4bd6-b493-f9420c59a074 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:23.589068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ff11d-33d4-4383-9eb7-1f2bc3698551 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:23.589458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ff11d-33d4-4383-9eb7-1f2bc3698551 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:23.589634 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ff11d-33d4-4383-9eb7-1f2bc3698551 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:23.591212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 314/613] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:04:23.597041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c661618-afad-4b49-8f35-2e912c6a8008 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:23.597320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c661618-afad-4b49-8f35-2e912c6a8008 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:23.597594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c661618-afad-4b49-8f35-2e912c6a8008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:23.617561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-7c661618-afad-4b49-8f35-2e912c6a8008 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:23.618742 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7c661618-afad-4b49-8f35-2e912c6a8008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:23.633027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:23.649421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df4c3f-58fb-4bd6-b493-f9420c59a074 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:23.721620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca71c3f-87e6-4e14-a76a-85ba77761b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:23.722151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca71c3f-87e6-4e14-a76a-85ba77761b54 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:23.722468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca71c3f-87e6-4e14-a76a-85ba77761b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:23.722749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca71c3f-87e6-4e14-a76a-85ba77761b54 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:23.723389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca71c3f-87e6-4e14-a76a-85ba77761b54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:23.723693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca71c3f-87e6-4e14-a76a-85ba77761b54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:23.725072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca71c3f-87e6-4e14-a76a-85ba77761b54 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:23.731318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 315/614] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:23.747406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 300/615] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:23.752917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c661618-afad-4b49-8f35-2e912c6a8008 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:23.900522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c661618-afad-4b49-8f35-2e912c6a8008 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:23.953902 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19776e66-d7b3-4edc-9721-938eb6985f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:23.954538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19776e66-d7b3-4edc-9721-938eb6985f0a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:23.954820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19776e66-d7b3-4edc-9721-938eb6985f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:23.955096 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19776e66-d7b3-4edc-9721-938eb6985f0a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:23.965551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19776e66-d7b3-4edc-9721-938eb6985f0a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:23.966994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19776e66-d7b3-4edc-9721-938eb6985f0a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-19776e66-d7b3-4edc-9721-938eb6985f0a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:23.983482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:23.984564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:23.985507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:23.985507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:23.985507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:23.985507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:23.985507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:23.985507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:23.985507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:23.985507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:23.985507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:23.985507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:23.986034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 316/616] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:23.992867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca71c3f-87e6-4e14-a76a-85ba77761b54 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:24.024236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:24.094623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19776e66-d7b3-4edc-9721-938eb6985f0a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:24.098079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:24.149615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca71c3f-87e6-4e14-a76a-85ba77761b54 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:24.185176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53ed00-889d-4374-a425-b56477bae06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:24.185839 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53ed00-889d-4374-a425-b56477bae06f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.186049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53ed00-889d-4374-a425-b56477bae06f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.186314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53ed00-889d-4374-a425-b56477bae06f None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.195797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53ed00-889d-4374-a425-b56477bae06f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:24.195797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53ed00-889d-4374-a425-b56477bae06f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:24.195797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53ed00-889d-4374-a425-b56477bae06f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:24.212852 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 301/617] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 7 16:04:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:24.215461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53ed00-889d-4374-a425-b56477bae06f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:24.314717 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19776e66-d7b3-4edc-9721-938eb6985f0a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:24.316512 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:24.381762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bd8a5-e85f-4745-9a9c-2b585429adc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:24.382261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53ed00-889d-4374-a425-b56477bae06f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:24.382374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bd8a5-e85f-4745-9a9c-2b585429adc3 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.382643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bd8a5-e85f-4745-9a9c-2b585429adc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.382915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bd8a5-e85f-4745-9a9c-2b585429adc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.396758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bd8a5-e85f-4745-9a9c-2b585429adc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-db6bd8a5-e85f-4745-9a9c-2b585429adc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:24.403583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:24.404881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:24.406040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 317/618] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:24.406898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed7a00-74be-4be0-a977-44a1faab5da1 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.406898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed7a00-74be-4be0-a977-44a1faab5da1 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.406898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed7a00-74be-4be0-a977-44a1faab5da1 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.412781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 318/619] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:24.413560 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed7a00-74be-4be0-a977-44a1faab5da1 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:24.428962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6afd7-c32c-45fd-8ee7-c9200262a63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:24.429652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6afd7-c32c-45fd-8ee7-c9200262a63f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.430037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6afd7-c32c-45fd-8ee7-c9200262a63f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.430153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6afd7-c32c-45fd-8ee7-c9200262a63f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.451578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6afd7-c32c-45fd-8ee7-c9200262a63f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:24.453819 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6afd7-c32c-45fd-8ee7-c9200262a63f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-88f6afd7-c32c-45fd-8ee7-c9200262a63f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:24.474585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:24.475627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:24.476562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:24.476562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:24.476562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:24.477491 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 302/620] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:24.478665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8ee78-0964-4b15-93b2-a288c1ca4675 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.478946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8ee78-0964-4b15-93b2-a288c1ca4675 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.479215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8ee78-0964-4b15-93b2-a288c1ca4675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.488452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:24.502379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-72b8ee78-0964-4b15-93b2-a288c1ca4675 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:24.504325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-72b8ee78-0964-4b15-93b2-a288c1ca4675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:24.531003 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed7a00-74be-4be0-a977-44a1faab5da1 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:24.568440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486bd59-bf45-4897-a214-f83fcbde2cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:24.569105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486bd59-bf45-4897-a214-f83fcbde2cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.569454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486bd59-bf45-4897-a214-f83fcbde2cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.569838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486bd59-bf45-4897-a214-f83fcbde2cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.570605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486bd59-bf45-4897-a214-f83fcbde2cf5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:24.570972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486bd59-bf45-4897-a214-f83fcbde2cf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:24.572511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486bd59-bf45-4897-a214-f83fcbde2cf5 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:24.586175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 319/621] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:24.587567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486bd59-bf45-4897-a214-f83fcbde2cf5 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:24.595490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 303/622] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:24.596478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8ee78-0964-4b15-93b2-a288c1ca4675 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:24.669329 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:24.725905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486bd59-bf45-4897-a214-f83fcbde2cf5 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:24.748896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8ee78-0964-4b15-93b2-a288c1ca4675 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:24.756976 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:24.768338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0f1b4-2f02-4325-957c-e96b70afa448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:24.775740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0f1b4-2f02-4325-957c-e96b70afa448 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.775740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0f1b4-2f02-4325-957c-e96b70afa448 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.775740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0f1b4-2f02-4325-957c-e96b70afa448 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.775740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0f1b4-2f02-4325-957c-e96b70afa448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:24.775740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0f1b4-2f02-4325-957c-e96b70afa448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:24.776147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b93a9-eec0-4d27-9e4c-49cbc9d0f3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:24.776241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0f1b4-2f02-4325-957c-e96b70afa448 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:24.776320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b93a9-eec0-4d27-9e4c-49cbc9d0f3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.776320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b93a9-eec0-4d27-9e4c-49cbc9d0f3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.776320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b93a9-eec0-4d27-9e4c-49cbc9d0f3e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.785338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b93a9-eec0-4d27-9e4c-49cbc9d0f3e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:24.786285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 320/623] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:24.787349 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b93a9-eec0-4d27-9e4c-49cbc9d0f3e0 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:24.793085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0f1b4-2f02-4325-957c-e96b70afa448 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-cb4b93a9-eec0-4d27-9e4c-49cbc9d0f3e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:24.804513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:24.805579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:24.806797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:24.806797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:24.806797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:24.809021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 304/624] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:24.814751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b93a9-eec0-4d27-9e4c-49cbc9d0f3e0 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:24.897513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0f1b4-2f02-4325-957c-e96b70afa448 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:24.922890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b93a9-eec0-4d27-9e4c-49cbc9d0f3e0 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:24.930019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:24.933741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed19728-3505-4fb3-a12c-30cee48d3001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:24.934447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed19728-3505-4fb3-a12c-30cee48d3001 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.934814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed19728-3505-4fb3-a12c-30cee48d3001 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.935161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed19728-3505-4fb3-a12c-30cee48d3001 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.942173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed19728-3505-4fb3-a12c-30cee48d3001 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:24.943865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed19728-3505-4fb3-a12c-30cee48d3001 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-eed19728-3505-4fb3-a12c-30cee48d3001 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:24.956106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:24.957027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:24.958187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:24.958187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:24.958187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:24.958187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:24.958187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:24.958187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:24.958187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:24.958187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:24.958187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 321/625] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:24.958887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb29420-1916-434a-a11a-098307312c8a None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.960974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb29420-1916-434a-a11a-098307312c8a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.960974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb29420-1916-434a-a11a-098307312c8a None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.961184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 322/626] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:24.963979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb29420-1916-434a-a11a-098307312c8a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:24.967435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe756af-8aeb-4929-ac5e-36bd8ac92583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:24.968106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe756af-8aeb-4929-ac5e-36bd8ac92583 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.968921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe756af-8aeb-4929-ac5e-36bd8ac92583 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.969197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe756af-8aeb-4929-ac5e-36bd8ac92583 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.969842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe756af-8aeb-4929-ac5e-36bd8ac92583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:24.970142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe756af-8aeb-4929-ac5e-36bd8ac92583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:24.971519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe756af-8aeb-4929-ac5e-36bd8ac92583 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:24.986139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 305/627] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:24.987371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6717b98-7153-45a3-ad3a-023627576e7d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.987650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6717b98-7153-45a3-ad3a-023627576e7d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.987942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6717b98-7153-45a3-ad3a-023627576e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.008887 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c6717b98-7153-45a3-ad3a-023627576e7d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:25.010793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c6717b98-7153-45a3-ad3a-023627576e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:25.049588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:25.089677 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 306/628] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:25.090951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6717b98-7153-45a3-ad3a-023627576e7d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:25.125663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb29420-1916-434a-a11a-098307312c8a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:25.176695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c14e07-5634-4749-b1e9-a686654a0287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:25.177808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c14e07-5634-4749-b1e9-a686654a0287 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:25.178704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c14e07-5634-4749-b1e9-a686654a0287 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:25.179156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c14e07-5634-4749-b1e9-a686654a0287 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.180306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c14e07-5634-4749-b1e9-a686654a0287 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:25.181044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c14e07-5634-4749-b1e9-a686654a0287 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:25.183306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c14e07-5634-4749-b1e9-a686654a0287 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:25.194399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:25.201818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 323/629] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:25.207465 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c14e07-5634-4749-b1e9-a686654a0287 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:25.207589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6717b98-7153-45a3-ad3a-023627576e7d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:25.244332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e725a-195a-495c-8d1a-13795bee450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:25.245115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e725a-195a-495c-8d1a-13795bee450c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:25.245490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e725a-195a-495c-8d1a-13795bee450c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:25.245863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e725a-195a-495c-8d1a-13795bee450c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.254889 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e725a-195a-495c-8d1a-13795bee450c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:25.256868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e725a-195a-495c-8d1a-13795bee450c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-c66e725a-195a-495c-8d1a-13795bee450c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:25.271666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:25.272858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:25.274025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:25.274025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:25.274025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:25.274025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:25.274025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:25.274025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:25.274025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:25.274025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:25.274025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:25.274025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:25.276254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 307/630] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 7 16:04:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:25.277285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e725a-195a-495c-8d1a-13795bee450c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:25.330097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c14e07-5634-4749-b1e9-a686654a0287 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:25.332801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:25.368277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8113fb-a6ee-4ec7-949d-985ca03c8b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:25.368947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8113fb-a6ee-4ec7-949d-985ca03c8b85 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:25.369228 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8113fb-a6ee-4ec7-949d-985ca03c8b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:25.369499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8113fb-a6ee-4ec7-949d-985ca03c8b85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.380741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8113fb-a6ee-4ec7-949d-985ca03c8b85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-2b8113fb-a6ee-4ec7-949d-985ca03c8b85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:25.385650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:25.386598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:25.387489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 324/631] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:25.388909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8113fb-a6ee-4ec7-949d-985ca03c8b85 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:25.397939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e725a-195a-495c-8d1a-13795bee450c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:25.434849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011505b4-d95e-41e2-b3f0-400e9ab42f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:25.435443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011505b4-d95e-41e2-b3f0-400e9ab42f87 None admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:25.435733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011505b4-d95e-41e2-b3f0-400e9ab42f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:25.436039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011505b4-d95e-41e2-b3f0-400e9ab42f87 None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/69a8796195ff4dfb85655cdd91a0229d/roles/d9477edba6304c829c6348bc4e6e6c1d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.453609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011505b4-d95e-41e2-b3f0-400e9ab42f87 None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=69a8796195ff4dfb85655cdd91a0229d, role_id=d9477edba6304c829c6348bc4e6e6c1d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:25.455117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011505b4-d95e-41e2-b3f0-400e9ab42f87 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:25.495934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8113fb-a6ee-4ec7-949d-985ca03c8b85 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:25.536285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 308/632] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:04:25 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/69a8796195ff4dfb85655cdd91a0229d/roles/d9477edba6304c829c6348bc4e6e6c1d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:25.537340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011505b4-d95e-41e2-b3f0-400e9ab42f87 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:25.541236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1c5ef-83ec-4be9-bc00-5b930bf5eede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:25.541890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1c5ef-83ec-4be9-bc00-5b930bf5eede None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:25.542177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1c5ef-83ec-4be9-bc00-5b930bf5eede None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:25.543589 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1c5ef-83ec-4be9-bc00-5b930bf5eede None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.544466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1c5ef-83ec-4be9-bc00-5b930bf5eede None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:25.544782 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1c5ef-83ec-4be9-bc00-5b930bf5eede None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:25.546280 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1c5ef-83ec-4be9-bc00-5b930bf5eede None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:25.551304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:25.570691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 325/633] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 7 16:04:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:25.571738 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1c5ef-83ec-4be9-bc00-5b930bf5eede None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:25.600234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:25.659687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011505b4-d95e-41e2-b3f0-400e9ab42f87 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:25.683064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1c5ef-83ec-4be9-bc00-5b930bf5eede None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:25.696745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c213dc-976b-4ea6-acf2-7bf6b155fba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:25.697309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c213dc-976b-4ea6-acf2-7bf6b155fba8 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:25.697591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c213dc-976b-4ea6-acf2-7bf6b155fba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:25.697865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c213dc-976b-4ea6-acf2-7bf6b155fba8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.713708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c213dc-976b-4ea6-acf2-7bf6b155fba8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-89c213dc-976b-4ea6-acf2-7bf6b155fba8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:25.718962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:25.720064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:25.721047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:25.723047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 309/634] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:25.724451 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbbd08-48cd-4a4d-b9d2-938a01e959ef None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:25.724919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88532ac4-f43f-45f4-a29f-0fd4cc9755bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:25.725140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbbd08-48cd-4a4d-b9d2-938a01e959ef None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:25.725531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88532ac4-f43f-45f4-a29f-0fd4cc9755bb None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:25.725695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbbd08-48cd-4a4d-b9d2-938a01e959ef None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.725797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88532ac4-f43f-45f4-a29f-0fd4cc9755bb None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:25.726076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88532ac4-f43f-45f4-a29f-0fd4cc9755bb None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.726752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88532ac4-f43f-45f4-a29f-0fd4cc9755bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:25.727059 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88532ac4-f43f-45f4-a29f-0fd4cc9755bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:25.727959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 310/635] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:25.728467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88532ac4-f43f-45f4-a29f-0fd4cc9755bb None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:25.729220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbbd08-48cd-4a4d-b9d2-938a01e959ef None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:25.739511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 326/636] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:25.740511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88532ac4-f43f-45f4-a29f-0fd4cc9755bb None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:25.814500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:25.826870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:25.843045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88532ac4-f43f-45f4-a29f-0fd4cc9755bb None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:25.853346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbbd08-48cd-4a4d-b9d2-938a01e959ef None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:25.880495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ed864-ace0-4f59-8966-99b7a5b3cab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:25.881097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ed864-ace0-4f59-8966-99b7a5b3cab6 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:25.881377 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ed864-ace0-4f59-8966-99b7a5b3cab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:25.881645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ed864-ace0-4f59-8966-99b7a5b3cab6 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.882293 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ed864-ace0-4f59-8966-99b7a5b3cab6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:25.882591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ed864-ace0-4f59-8966-99b7a5b3cab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:25.883962 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201ed864-ace0-4f59-8966-99b7a5b3cab6 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:25.890803 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0970707-43ac-4382-9ddb-c09d864cf63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:25.891507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0970707-43ac-4382-9ddb-c09d864cf63c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:25.891838 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0970707-43ac-4382-9ddb-c09d864cf63c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:25.892119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0970707-43ac-4382-9ddb-c09d864cf63c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.897542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 327/637] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:25.897542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06625d3e-74a9-440a-84bd-06d4ab51f8b4 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:25.897542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06625d3e-74a9-440a-84bd-06d4ab51f8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:25.897542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06625d3e-74a9-440a-84bd-06d4ab51f8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:25.912685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-06625d3e-74a9-440a-84bd-06d4ab51f8b4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:25.914009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-06625d3e-74a9-440a-84bd-06d4ab51f8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:25.971839 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0970707-43ac-4382-9ddb-c09d864cf63c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:25.973239 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0970707-43ac-4382-9ddb-c09d864cf63c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:25.983855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 328/638] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:25.985227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06625d3e-74a9-440a-84bd-06d4ab51f8b4 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:26.066253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:26.080566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:26.105341 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06625d3e-74a9-440a-84bd-06d4ab51f8b4 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:26.115851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 311/639] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:26.115851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0970707-43ac-4382-9ddb-c09d864cf63c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:26.149134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de09f9-fdba-49a9-8f7d-b60c30377bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:26.150416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de09f9-fdba-49a9-8f7d-b60c30377bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:26.150873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de09f9-fdba-49a9-8f7d-b60c30377bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:26.151340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de09f9-fdba-49a9-8f7d-b60c30377bb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:26.163004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de09f9-fdba-49a9-8f7d-b60c30377bb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:26.164823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de09f9-fdba-49a9-8f7d-b60c30377bb7 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-89de09f9-fdba-49a9-8f7d-b60c30377bb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:26.182150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:26.183219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:26.184151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:26.184151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:26.184151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:26.185932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 329/640] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:26.187904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de09f9-fdba-49a9-8f7d-b60c30377bb7 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:26.235284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0970707-43ac-4382-9ddb-c09d864cf63c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:26.275641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e3b81-5103-4e8f-be7e-b8222d9358e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:26.276305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e3b81-5103-4e8f-be7e-b8222d9358e6 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:26.276594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e3b81-5103-4e8f-be7e-b8222d9358e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:26.276870 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82e3b81-5103-4e8f-be7e-b8222d9358e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:26.287799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e3b81-5103-4e8f-be7e-b8222d9358e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:26.289257 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82e3b81-5103-4e8f-be7e-b8222d9358e6 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:26.298384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de09f9-fdba-49a9-8f7d-b60c30377bb7 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-f82e3b81-5103-4e8f-be7e-b8222d9358e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:26.305117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:26.306372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:26.307307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:26.307307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:26.307307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:26.307307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 312/641] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:26.307710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e3b81-5103-4e8f-be7e-b8222d9358e6 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:26.329356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:26.330060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:26.338334 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db3725-d68e-4ad8-9447-69373077b354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:26.343881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db3725-d68e-4ad8-9447-69373077b354 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:26.343881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db3725-d68e-4ad8-9447-69373077b354 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:26.343881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db3725-d68e-4ad8-9447-69373077b354 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:26.351322 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db3725-d68e-4ad8-9447-69373077b354 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:26.352850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db3725-d68e-4ad8-9447-69373077b354 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-90db3725-d68e-4ad8-9447-69373077b354 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:26.368600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:26.373029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:26.374099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:26.374099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:26.374099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:26.374099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 330/642] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:26.374099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db3725-d68e-4ad8-9447-69373077b354 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:26.429890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82e3b81-5103-4e8f-be7e-b8222d9358e6 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:26.466330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d2763-d653-4cb6-b2db-b090a058060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:26.467101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d2763-d653-4cb6-b2db-b090a058060a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:26.467516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d2763-d653-4cb6-b2db-b090a058060a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:26.467879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d2763-d653-4cb6-b2db-b090a058060a None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:26.469058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d2763-d653-4cb6-b2db-b090a058060a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:26.469358 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d2763-d653-4cb6-b2db-b090a058060a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:26.470877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3d2763-d653-4cb6-b2db-b090a058060a None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:26.482374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 313/643] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:26.483154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d2763-d653-4cb6-b2db-b090a058060a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:26.508028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db3725-d68e-4ad8-9447-69373077b354 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:26.547201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed8b3f-221e-43a5-967c-91f761eb5c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:26.547818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8b3f-221e-43a5-967c-91f761eb5c53 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:26.548101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8b3f-221e-43a5-967c-91f761eb5c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:26.548402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ed8b3f-221e-43a5-967c-91f761eb5c53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:26.559538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ed8b3f-221e-43a5-967c-91f761eb5c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-37ed8b3f-221e-43a5-967c-91f761eb5c53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:26.564854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:26.565794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 331/644] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:26.567264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed8b3f-221e-43a5-967c-91f761eb5c53 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:26.574328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:26.596439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d2763-d653-4cb6-b2db-b090a058060a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:26.610576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:26.634644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5b7e1-c145-443d-a3ee-0ed09e6a56a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:26.635259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5b7e1-c145-443d-a3ee-0ed09e6a56a3 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:26.635543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5b7e1-c145-443d-a3ee-0ed09e6a56a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:26.635841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5b7e1-c145-443d-a3ee-0ed09e6a56a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:26.636487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5b7e1-c145-443d-a3ee-0ed09e6a56a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:26.636788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5b7e1-c145-443d-a3ee-0ed09e6a56a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:26.638316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5b7e1-c145-443d-a3ee-0ed09e6a56a3 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:26.656754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 314/645] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:26.656754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5b7e1-c145-443d-a3ee-0ed09e6a56a3 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:26.683055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ed8b3f-221e-43a5-967c-91f761eb5c53 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:26.720250 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4ca16-e76a-4e9c-95df-04d95e9bf2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:26.720964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4ca16-e76a-4e9c-95df-04d95e9bf2c9 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:26.721258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4ca16-e76a-4e9c-95df-04d95e9bf2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:26.721578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4ca16-e76a-4e9c-95df-04d95e9bf2c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:26.722219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4ca16-e76a-4e9c-95df-04d95e9bf2c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:26.722576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4ca16-e76a-4e9c-95df-04d95e9bf2c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:26.723978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4ca16-e76a-4e9c-95df-04d95e9bf2c9 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:26.742346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 332/646] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:26.742346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4ca16-e76a-4e9c-95df-04d95e9bf2c9 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:26.775941 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5b7e1-c145-443d-a3ee-0ed09e6a56a3 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:26.813325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94fb0-aa8d-4020-b621-cc0c2d275a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:26.814129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94fb0-aa8d-4020-b621-cc0c2d275a42 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:26.814527 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94fb0-aa8d-4020-b621-cc0c2d275a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:26.814869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f94fb0-aa8d-4020-b621-cc0c2d275a42 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:26.822482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f94fb0-aa8d-4020-b621-cc0c2d275a42 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:26.824062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f94fb0-aa8d-4020-b621-cc0c2d275a42 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:26.825759 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-05f94fb0-aa8d-4020-b621-cc0c2d275a42 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:26.836380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:26.843259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 315/647] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:26.844956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94fb0-aa8d-4020-b621-cc0c2d275a42 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:26.869227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:26.872586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4ca16-e76a-4e9c-95df-04d95e9bf2c9 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:26.908026 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9fae-a803-44f5-80d5-43d35a51e1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:26.908647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9fae-a803-44f5-80d5-43d35a51e1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:26.908950 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9fae-a803-44f5-80d5-43d35a51e1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:26.909233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff9fae-a803-44f5-80d5-43d35a51e1c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:26.910107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9fae-a803-44f5-80d5-43d35a51e1c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:26.910716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9fae-a803-44f5-80d5-43d35a51e1c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:26.912113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ff9fae-a803-44f5-80d5-43d35a51e1c6 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:26.920590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 333/648] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:26.921400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9fae-a803-44f5-80d5-43d35a51e1c6 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:26.948219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f94fb0-aa8d-4020-b621-cc0c2d275a42 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:26.984894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e9724-54fa-469f-acd3-8408f342a489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:26.985525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e9724-54fa-469f-acd3-8408f342a489 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:26.985812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e9724-54fa-469f-acd3-8408f342a489 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:26.986146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e9724-54fa-469f-acd3-8408f342a489 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:26.996490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e9724-54fa-469f-acd3-8408f342a489 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:26.997963 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e9724-54fa-469f-acd3-8408f342a489 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-788e9724-54fa-469f-acd3-8408f342a489 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:27.014094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:27.015024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:27.016574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:27.016574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:27.016574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:27.016574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:27.016574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:27.016574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:27.016574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:27.016574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:27.016574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 16:04:27.016574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:27.019909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 316/649] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:27.020938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e9724-54fa-469f-acd3-8408f342a489 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:27.039247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff9fae-a803-44f5-80d5-43d35a51e1c6 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:27.074474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dc22c-186b-4555-9341-3f22187afda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:27.075107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dc22c-186b-4555-9341-3f22187afda9 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:27.075448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dc22c-186b-4555-9341-3f22187afda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:27.075731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5dc22c-186b-4555-9341-3f22187afda9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:27.083078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5dc22c-186b-4555-9341-3f22187afda9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:27.084591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5dc22c-186b-4555-9341-3f22187afda9 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:27.086616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-1c5dc22c-186b-4555-9341-3f22187afda9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:27.097824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:27.098924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:27.099858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:27.099858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:27.099858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:27.099858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:27.099858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:27.099858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:27.099858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:27.099858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:27.101028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 334/650] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:27.102045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dc22c-186b-4555-9341-3f22187afda9 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:27.132904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:27.142185 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e9724-54fa-469f-acd3-8408f342a489 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:27.182574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979b36e-fb60-428a-ada6-74bfa152caec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:27.183193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979b36e-fb60-428a-ada6-74bfa152caec None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:27.183476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979b36e-fb60-428a-ada6-74bfa152caec None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:27.183756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0979b36e-fb60-428a-ada6-74bfa152caec None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:27.184576 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979b36e-fb60-428a-ada6-74bfa152caec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:27.184887 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979b36e-fb60-428a-ada6-74bfa152caec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:27.186497 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0979b36e-fb60-428a-ada6-74bfa152caec None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:27.211193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 317/651] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 7 16:04:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:27.211193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979b36e-fb60-428a-ada6-74bfa152caec None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:27.216961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5dc22c-186b-4555-9341-3f22187afda9 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:27.250293 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd415e-a038-433b-b556-92b960064b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:27.250923 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd415e-a038-433b-b556-92b960064b4c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:27.251204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd415e-a038-433b-b556-92b960064b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:27.251480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd415e-a038-433b-b556-92b960064b4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:27.261928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd415e-a038-433b-b556-92b960064b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:27.263374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd415e-a038-433b-b556-92b960064b4c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-37bd415e-a038-433b-b556-92b960064b4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:27.277865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:27.278918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:27.279863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:27.279863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:27.279863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:27.279863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 335/652] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:27.279863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd415e-a038-433b-b556-92b960064b4c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:27.318583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0979b36e-fb60-428a-ada6-74bfa152caec None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:27.340674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:27.360216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab113faf-b46d-4d61-a9a8-66674e033fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:27.360827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab113faf-b46d-4d61-a9a8-66674e033fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:27.361126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab113faf-b46d-4d61-a9a8-66674e033fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:27.361432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab113faf-b46d-4d61-a9a8-66674e033fb8 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:27.362124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab113faf-b46d-4d61-a9a8-66674e033fb8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:27.362426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab113faf-b46d-4d61-a9a8-66674e033fb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:27.363391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:27.363954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab113faf-b46d-4d61-a9a8-66674e033fb8 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:27.385931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 318/653] 10.222.0.15 () {56 vars in 1137 bytes} [Wed Sep 7 16:04:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:27.385931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab113faf-b46d-4d61-a9a8-66674e033fb8 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:27.397342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd415e-a038-433b-b556-92b960064b4c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:27.441343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfd35f-3759-43d2-a35e-f3f77be3076f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:27.442060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfd35f-3759-43d2-a35e-f3f77be3076f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:27.442341 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfd35f-3759-43d2-a35e-f3f77be3076f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:27.442618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfd35f-3759-43d2-a35e-f3f77be3076f None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:27.443309 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfd35f-3759-43d2-a35e-f3f77be3076f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:27.443617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfd35f-3759-43d2-a35e-f3f77be3076f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:27.445090 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfd35f-3759-43d2-a35e-f3f77be3076f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:27.463209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 336/654] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:27.463209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfd35f-3759-43d2-a35e-f3f77be3076f None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:27.490662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab113faf-b46d-4d61-a9a8-66674e033fb8 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:27.528288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f829c4-8e22-4285-b9e0-c97b1276d552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:27.528899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f829c4-8e22-4285-b9e0-c97b1276d552 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:27.529180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f829c4-8e22-4285-b9e0-c97b1276d552 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:27.529450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f829c4-8e22-4285-b9e0-c97b1276d552 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:27.568940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfd35f-3759-43d2-a35e-f3f77be3076f None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:27.574268 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:27.600168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f829c4-8e22-4285-b9e0-c97b1276d552 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:27.601728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f829c4-8e22-4285-b9e0-c97b1276d552 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:27.605681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9b52a-220b-44ec-8416-a7996bb6c8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:27.606644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9b52a-220b-44ec-8416-a7996bb6c8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:27.607074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9b52a-220b-44ec-8416-a7996bb6c8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:27.607500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9b52a-220b-44ec-8416-a7996bb6c8e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:27.608194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9b52a-220b-44ec-8416-a7996bb6c8e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:27.608500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9b52a-220b-44ec-8416-a7996bb6c8e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:27.610495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9b52a-220b-44ec-8416-a7996bb6c8e3 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:27.613623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:27.628780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 337/655] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:27.628780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9b52a-220b-44ec-8416-a7996bb6c8e3 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:27.715934 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 319/656] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:27.716767 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f829c4-8e22-4285-b9e0-c97b1276d552 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:27.730381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9b52a-220b-44ec-8416-a7996bb6c8e3 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:27.763743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f2e13a-e0fe-413e-91ee-5039c1eeb9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:27.764380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2e13a-e0fe-413e-91ee-5039c1eeb9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:27.764660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2e13a-e0fe-413e-91ee-5039c1eeb9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:27.764941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2e13a-e0fe-413e-91ee-5039c1eeb9b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:27.789422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:27.813768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f829c4-8e22-4285-b9e0-c97b1276d552 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:27.824447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:27.831891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f2e13a-e0fe-413e-91ee-5039c1eeb9b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:27.833336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f2e13a-e0fe-413e-91ee-5039c1eeb9b0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:27.847392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd0953-d3b0-4293-8b40-ec43945b2bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:27.848018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd0953-d3b0-4293-8b40-ec43945b2bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:27.848297 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd0953-d3b0-4293-8b40-ec43945b2bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:27.848573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd0953-d3b0-4293-8b40-ec43945b2bb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:27.849293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd0953-d3b0-4293-8b40-ec43945b2bb6 None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=45061db68b1e40f59935cc9a46d2ff10) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:27.849604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd0953-d3b0-4293-8b40-ec43945b2bb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:27.851007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd0953-d3b0-4293-8b40-ec43945b2bb6 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:27.856675 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 320/657] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:27 2022] GET /identity/v3/role_assignments?user.id=45061db68b1e40f59935cc9a46d2ff10&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:27.857525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd0953-d3b0-4293-8b40-ec43945b2bb6 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:27.943271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 338/658] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:27.943271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f5e7a-3dfa-4da6-ad31-d41a1fc67d19 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:27.943271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f5e7a-3dfa-4da6-ad31-d41a1fc67d19 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:27.943271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681f5e7a-3dfa-4da6-ad31-d41a1fc67d19 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:27.943271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 339/659] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:27.943271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f5e7a-3dfa-4da6-ad31-d41a1fc67d19 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:28.009264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd0953-d3b0-4293-8b40-ec43945b2bb6 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:28.052209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de71bd-dca9-49e6-84d3-1cf582a849b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:28.052963 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de71bd-dca9-49e6-84d3-1cf582a849b9 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:28.053318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de71bd-dca9-49e6-84d3-1cf582a849b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:28.053591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de71bd-dca9-49e6-84d3-1cf582a849b9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:28.054767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:28.061973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de71bd-dca9-49e6-84d3-1cf582a849b9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:28.062956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681f5e7a-3dfa-4da6-ad31-d41a1fc67d19 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:28.063426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de71bd-dca9-49e6-84d3-1cf582a849b9 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:28.065383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-06de71bd-dca9-49e6-84d3-1cf582a849b9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:28.075697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:28.076976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:28.077967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:28.077967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:28.077967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:28.077967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:28.077967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:28.077967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:28.077967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:28.077967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:28.077967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 16:04:28.077967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:28.079833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 321/660] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:28.080777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de71bd-dca9-49e6-84d3-1cf582a849b9 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:28.100221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892eade0-54f3-4cc4-80bf-8d399fedf6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:28.101531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eade0-54f3-4cc4-80bf-8d399fedf6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:28.101531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eade0-54f3-4cc4-80bf-8d399fedf6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:28.101531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892eade0-54f3-4cc4-80bf-8d399fedf6d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:28.116877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892eade0-54f3-4cc4-80bf-8d399fedf6d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:28.118348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892eade0-54f3-4cc4-80bf-8d399fedf6d6 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-892eade0-54f3-4cc4-80bf-8d399fedf6d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:28.134498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:28.135495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:28.136436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:28.136436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:28.136436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:28.136436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 340/661] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:28.137183 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d358d1-162e-40cc-a7f7-df0d595d30f0 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:28.137450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d358d1-162e-40cc-a7f7-df0d595d30f0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:28.137719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d358d1-162e-40cc-a7f7-df0d595d30f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:28.157239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-89d358d1-162e-40cc-a7f7-df0d595d30f0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:28.158444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-89d358d1-162e-40cc-a7f7-df0d595d30f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:28.207524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de71bd-dca9-49e6-84d3-1cf582a849b9 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:28.229926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 341/662] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:28.231330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d358d1-162e-40cc-a7f7-df0d595d30f0 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:28.247786 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc742411-0eb8-4b25-a952-1d161ae95d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:28.248894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc742411-0eb8-4b25-a952-1d161ae95d9d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:28.249657 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc742411-0eb8-4b25-a952-1d161ae95d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:28.250109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc742411-0eb8-4b25-a952-1d161ae95d9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:28.264181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc742411-0eb8-4b25-a952-1d161ae95d9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:28.267447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc742411-0eb8-4b25-a952-1d161ae95d9d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-bc742411-0eb8-4b25-a952-1d161ae95d9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:28.285766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:28.286730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:28.287610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:28.287610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:28.287610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:28.289014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 322/663] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:28.289840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc742411-0eb8-4b25-a952-1d161ae95d9d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:28.343014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d358d1-162e-40cc-a7f7-df0d595d30f0 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:28.353086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:28.360860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:28.386410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0254d9-47b9-4cca-8401-5ecf3d6914e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:28.387030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0254d9-47b9-4cca-8401-5ecf3d6914e6 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:28.387492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0254d9-47b9-4cca-8401-5ecf3d6914e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:28.388431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0254d9-47b9-4cca-8401-5ecf3d6914e6 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:28.389190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0254d9-47b9-4cca-8401-5ecf3d6914e6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:28.389503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0254d9-47b9-4cca-8401-5ecf3d6914e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:28.392945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0254d9-47b9-4cca-8401-5ecf3d6914e6 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:28.413938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 342/664] 10.222.0.15 () {56 vars in 1139 bytes} [Wed Sep 7 16:04:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:28.415121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0254d9-47b9-4cca-8401-5ecf3d6914e6 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:28.420128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc742411-0eb8-4b25-a952-1d161ae95d9d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:28.456169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee6482-44cd-4bd8-bb32-07a8afbee3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:28.456782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee6482-44cd-4bd8-bb32-07a8afbee3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:28.457067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee6482-44cd-4bd8-bb32-07a8afbee3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:28.457347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee6482-44cd-4bd8-bb32-07a8afbee3e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:28.457998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee6482-44cd-4bd8-bb32-07a8afbee3e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:28.458303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee6482-44cd-4bd8-bb32-07a8afbee3e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:28.459669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee6482-44cd-4bd8-bb32-07a8afbee3e4 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:28.473106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 323/665] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:28.473744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee6482-44cd-4bd8-bb32-07a8afbee3e4 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:28.527103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0254d9-47b9-4cca-8401-5ecf3d6914e6 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:28.562517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f96b14-aa90-45bc-be20-82fc2e063804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:28.563121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f96b14-aa90-45bc-be20-82fc2e063804 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:28.563397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f96b14-aa90-45bc-be20-82fc2e063804 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:28.563670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f96b14-aa90-45bc-be20-82fc2e063804 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:28.564327 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f96b14-aa90-45bc-be20-82fc2e063804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:28.564635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f96b14-aa90-45bc-be20-82fc2e063804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:28.565993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f96b14-aa90-45bc-be20-82fc2e063804 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:28.567427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:28.579549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 343/666] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:28.580544 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f96b14-aa90-45bc-be20-82fc2e063804 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:28.587101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee6482-44cd-4bd8-bb32-07a8afbee3e4 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:28.600081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:28.624841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab084b1-3cf3-410e-81c9-de2b848eebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:28.625934 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab084b1-3cf3-410e-81c9-de2b848eebea None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:28.626634 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab084b1-3cf3-410e-81c9-de2b848eebea None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:28.627167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab084b1-3cf3-410e-81c9-de2b848eebea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:28.690297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f96b14-aa90-45bc-be20-82fc2e063804 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:28.704968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab084b1-3cf3-410e-81c9-de2b848eebea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:28.706454 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab084b1-3cf3-410e-81c9-de2b848eebea None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:28.726790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a74b7-2f62-486c-b76c-8be7d997d8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:28.727525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a74b7-2f62-486c-b76c-8be7d997d8dd None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:28.727835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a74b7-2f62-486c-b76c-8be7d997d8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:28.728111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a74b7-2f62-486c-b76c-8be7d997d8dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:28.735231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a74b7-2f62-486c-b76c-8be7d997d8dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:28.736693 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a74b7-2f62-486c-b76c-8be7d997d8dd None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-214a74b7-2f62-486c-b76c-8be7d997d8dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:28.748002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:28.748938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:28.750420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:28.750420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:28.750420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:28.750420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:28.750420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:28.750420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:28.750420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:28.750420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:28.754247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 344/667] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:28.755076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a74b7-2f62-486c-b76c-8be7d997d8dd None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:28.815206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 324/668] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:28.815206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab084b1-3cf3-410e-81c9-de2b848eebea None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:28.816503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:28.826828 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:28.864349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a74b7-2f62-486c-b76c-8be7d997d8dd None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:28.905440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97449c97-2a92-4d2a-a2ff-3f21b89c295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:28.906103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97449c97-2a92-4d2a-a2ff-3f21b89c295a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:28.906639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97449c97-2a92-4d2a-a2ff-3f21b89c295a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:28.906976 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97449c97-2a92-4d2a-a2ff-3f21b89c295a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:28.917741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97449c97-2a92-4d2a-a2ff-3f21b89c295a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:28.922662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab084b1-3cf3-410e-81c9-de2b848eebea None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-97449c97-2a92-4d2a-a2ff-3f21b89c295a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:28.923645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:28.924567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:28.925404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 345/669] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:28.926087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97449c97-2a92-4d2a-a2ff-3f21b89c295a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:28.970023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808612ed-11cc-4b8b-b495-4d8a6f678b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:28.970968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808612ed-11cc-4b8b-b495-4d8a6f678b44 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:28.971416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808612ed-11cc-4b8b-b495-4d8a6f678b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:28.971699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808612ed-11cc-4b8b-b495-4d8a6f678b44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:28.978428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808612ed-11cc-4b8b-b495-4d8a6f678b44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:28.979887 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808612ed-11cc-4b8b-b495-4d8a6f678b44 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-808612ed-11cc-4b8b-b495-4d8a6f678b44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:28.991639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:28.992768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:28.993824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:28.993824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:28.993824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:28.993824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:28.993824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:28.993824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:28.993824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:28.993824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:29.002920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 325/670] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:29.002920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808612ed-11cc-4b8b-b495-4d8a6f678b44 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.037197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97449c97-2a92-4d2a-a2ff-3f21b89c295a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.053190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:29.075041 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:29.076292 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd1159-a68c-4eff-bc67-1704a558712c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.076907 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd1159-a68c-4eff-bc67-1704a558712c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.077186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd1159-a68c-4eff-bc67-1704a558712c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.077489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd1159-a68c-4eff-bc67-1704a558712c None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.078176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd1159-a68c-4eff-bc67-1704a558712c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:29.078487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd1159-a68c-4eff-bc67-1704a558712c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.080058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dd1159-a68c-4eff-bc67-1704a558712c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.093443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 346/671] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:29.094252 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd1159-a68c-4eff-bc67-1704a558712c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.108451 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808612ed-11cc-4b8b-b495-4d8a6f678b44 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.146214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee543d22-a129-4488-9d74-0b8c7ac22ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.146766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee543d22-a129-4488-9d74-0b8c7ac22ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.147035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee543d22-a129-4488-9d74-0b8c7ac22ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.147313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee543d22-a129-4488-9d74-0b8c7ac22ff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.157337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee543d22-a129-4488-9d74-0b8c7ac22ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.158793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee543d22-a129-4488-9d74-0b8c7ac22ff3 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-ee543d22-a129-4488-9d74-0b8c7ac22ff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:29.173811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:29.174782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:29.175679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:29.175679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:29.175679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:29.175679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 326/672] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:29.175679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee543d22-a129-4488-9d74-0b8c7ac22ff3 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.219313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd1159-a68c-4eff-bc67-1704a558712c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.255163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c971ea5-aaf0-49ea-b556-5a62ac7ea3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.255940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c971ea5-aaf0-49ea-b556-5a62ac7ea3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.256298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c971ea5-aaf0-49ea-b556-5a62ac7ea3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.256580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c971ea5-aaf0-49ea-b556-5a62ac7ea3d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.257247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c971ea5-aaf0-49ea-b556-5a62ac7ea3d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:29.257555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c971ea5-aaf0-49ea-b556-5a62ac7ea3d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.258975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c971ea5-aaf0-49ea-b556-5a62ac7ea3d2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.266841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 347/673] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:29.268304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c971ea5-aaf0-49ea-b556-5a62ac7ea3d2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.291850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee543d22-a129-4488-9d74-0b8c7ac22ff3 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.308309 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:29.317603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:29.329935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36571de-3832-4353-abb2-4b01044aa36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.335885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36571de-3832-4353-abb2-4b01044aa36a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.335885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36571de-3832-4353-abb2-4b01044aa36a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.335885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36571de-3832-4353-abb2-4b01044aa36a None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.335885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36571de-3832-4353-abb2-4b01044aa36a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:29.335885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36571de-3832-4353-abb2-4b01044aa36a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.335885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36571de-3832-4353-abb2-4b01044aa36a None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.351342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 327/674] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:29.351342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36571de-3832-4353-abb2-4b01044aa36a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.379199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c971ea5-aaf0-49ea-b556-5a62ac7ea3d2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.414278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cda7dd-5e3c-42ec-98c4-b5bec28972a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.414809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cda7dd-5e3c-42ec-98c4-b5bec28972a9 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.415087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cda7dd-5e3c-42ec-98c4-b5bec28972a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.415360 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cda7dd-5e3c-42ec-98c4-b5bec28972a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.416122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cda7dd-5e3c-42ec-98c4-b5bec28972a9 None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=0a5d7cf7534941cb857a4ab5867d978e) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:29.416425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cda7dd-5e3c-42ec-98c4-b5bec28972a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.417846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cda7dd-5e3c-42ec-98c4-b5bec28972a9 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.424087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 348/675] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/role_assignments?user.id=0a5d7cf7534941cb857a4ab5867d978e&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:29.424854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cda7dd-5e3c-42ec-98c4-b5bec28972a9 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.457812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36571de-3832-4353-abb2-4b01044aa36a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.495293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e16d8-6900-48f8-9dca-3df829aa4710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.496042 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e16d8-6900-48f8-9dca-3df829aa4710 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.496391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e16d8-6900-48f8-9dca-3df829aa4710 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.496728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e16d8-6900-48f8-9dca-3df829aa4710 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.497462 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e16d8-6900-48f8-9dca-3df829aa4710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:29.497833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e16d8-6900-48f8-9dca-3df829aa4710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.499339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246e16d8-6900-48f8-9dca-3df829aa4710 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.516180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 328/676] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:29.516180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e16d8-6900-48f8-9dca-3df829aa4710 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.567945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:29.573085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:29.576942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cda7dd-5e3c-42ec-98c4-b5bec28972a9 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.618110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4323ae3-56a6-4bf2-b6fc-85785a2b4b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.619046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4323ae3-56a6-4bf2-b6fc-85785a2b4b74 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.619481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4323ae3-56a6-4bf2-b6fc-85785a2b4b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.619934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4323ae3-56a6-4bf2-b6fc-85785a2b4b74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.630358 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e16d8-6900-48f8-9dca-3df829aa4710 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.632173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4323ae3-56a6-4bf2-b6fc-85785a2b4b74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.633963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4323ae3-56a6-4bf2-b6fc-85785a2b4b74 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-a4323ae3-56a6-4bf2-b6fc-85785a2b4b74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:29.654760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:29.656542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:29.657440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:29.657440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:29.657440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:29.657440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 349/677] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:29.657440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4323ae3-56a6-4bf2-b6fc-85785a2b4b74 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.675473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1726e-cfff-4542-88a4-08f4520ca397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.676454 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1726e-cfff-4542-88a4-08f4520ca397 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.676941 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1726e-cfff-4542-88a4-08f4520ca397 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.677315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c1726e-cfff-4542-88a4-08f4520ca397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.678153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1726e-cfff-4542-88a4-08f4520ca397 None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=9f5f8dfbb9c44e788f110a4f53425344) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:29.678564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1726e-cfff-4542-88a4-08f4520ca397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.681388 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c1726e-cfff-4542-88a4-08f4520ca397 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.687115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 329/678] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/role_assignments?user.id=9f5f8dfbb9c44e788f110a4f53425344&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:29.689343 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1726e-cfff-4542-88a4-08f4520ca397 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.806321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4323ae3-56a6-4bf2-b6fc-85785a2b4b74 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.815105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c1726e-cfff-4542-88a4-08f4520ca397 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.824382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:29.855258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e41cdb-d3ee-4327-95b7-7441cda13e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.856030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e41cdb-d3ee-4327-95b7-7441cda13e69 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.856356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d412c5-63a2-4817-a17c-1c1669d58869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.856575 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e41cdb-d3ee-4327-95b7-7441cda13e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.856950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e41cdb-d3ee-4327-95b7-7441cda13e69 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.857494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d412c5-63a2-4817-a17c-1c1669d58869 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.857946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d412c5-63a2-4817-a17c-1c1669d58869 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.858305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e41cdb-d3ee-4327-95b7-7441cda13e69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:29.858513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d412c5-63a2-4817-a17c-1c1669d58869 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.858837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e41cdb-d3ee-4327-95b7-7441cda13e69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.860564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e41cdb-d3ee-4327-95b7-7441cda13e69 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.869491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d412c5-63a2-4817-a17c-1c1669d58869 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.871938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d412c5-63a2-4817-a17c-1c1669d58869 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.879501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 330/679] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:29.884936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e41cdb-d3ee-4327-95b7-7441cda13e69 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-32d412c5-63a2-4817-a17c-1c1669d58869 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:29.887816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:29.888936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:29.890615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:29.890615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:29.890615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:29.890615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:29.890615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:29.890615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:29.890615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:29.890615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:29.892324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 350/680] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:29.898253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d412c5-63a2-4817-a17c-1c1669d58869 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.917199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:30.025994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e41cdb-d3ee-4327-95b7-7441cda13e69 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:30.035158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d412c5-63a2-4817-a17c-1c1669d58869 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:30.070271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854cf9c6-f11c-4bc7-bb4a-4e03a6642346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:30.071086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cf9c6-f11c-4bc7-bb4a-4e03a6642346 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:30.071469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cf9c6-f11c-4bc7-bb4a-4e03a6642346 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:30.071953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cf9c6-f11c-4bc7-bb4a-4e03a6642346 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:30.076298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431649bf-27b4-45f4-a9b2-49b47527da76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:30.077116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431649bf-27b4-45f4-a9b2-49b47527da76 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:30.078113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431649bf-27b4-45f4-a9b2-49b47527da76 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:30.078537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431649bf-27b4-45f4-a9b2-49b47527da76 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:30.079399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431649bf-27b4-45f4-a9b2-49b47527da76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:30.079877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431649bf-27b4-45f4-a9b2-49b47527da76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:30.081332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854cf9c6-f11c-4bc7-bb4a-4e03a6642346 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:30.081671 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431649bf-27b4-45f4-a9b2-49b47527da76 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:30.082961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854cf9c6-f11c-4bc7-bb4a-4e03a6642346 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:30.095233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 351/681] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-854cf9c6-f11c-4bc7-bb4a-4e03a6642346 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:30.099827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:30.100792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:30.101754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:30.101754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:30.101754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:30.101754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:30.101754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:30.101754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:30.101754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:30.101754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:30.101754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:30.101754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:30.101754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 331/682] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:30.103295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431649bf-27b4-45f4-a9b2-49b47527da76 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:30.103581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854cf9c6-f11c-4bc7-bb4a-4e03a6642346 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:30.134957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:30.189155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:30.221209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431649bf-27b4-45f4-a9b2-49b47527da76 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:30.225906 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854cf9c6-f11c-4bc7-bb4a-4e03a6642346 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:30.259162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1669e9-e45e-4fc0-a5c8-89f76305e0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:30.260384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1669e9-e45e-4fc0-a5c8-89f76305e0da None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:30.260869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1669e9-e45e-4fc0-a5c8-89f76305e0da None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:30.261282 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1669e9-e45e-4fc0-a5c8-89f76305e0da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:30.262224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1669e9-e45e-4fc0-a5c8-89f76305e0da None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=b6a6b52462624e7f84f479144353b51e) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:30.262681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1669e9-e45e-4fc0-a5c8-89f76305e0da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:30.264317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1669e9-e45e-4fc0-a5c8-89f76305e0da None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:30.270901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8df969-4f40-4fe4-a35f-0ad7dd4d3cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:30.271643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8df969-4f40-4fe4-a35f-0ad7dd4d3cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:30.272051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8df969-4f40-4fe4-a35f-0ad7dd4d3cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:30.272440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8df969-4f40-4fe4-a35f-0ad7dd4d3cf5 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:30.273110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 352/683] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:30 2022] GET /identity/v3/role_assignments?user.id=b6a6b52462624e7f84f479144353b51e&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:30.274535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c0f4-f8df-48e1-bd09-c5211e542af2 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:30.274953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c0f4-f8df-48e1-bd09-c5211e542af2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:30.275358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae6c0f4-f8df-48e1-bd09-c5211e542af2 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:30.281269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 353/684] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:30.281913 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8df969-4f40-4fe4-a35f-0ad7dd4d3cf5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:30.282436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8df969-4f40-4fe4-a35f-0ad7dd4d3cf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:30.283640 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4882c73-4ec1-4d83-95e4-dd157009c812 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:30.284114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4882c73-4ec1-4d83-95e4-dd157009c812 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:30.284229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8df969-4f40-4fe4-a35f-0ad7dd4d3cf5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:30.284652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4882c73-4ec1-4d83-95e4-dd157009c812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:30.305972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-a4882c73-4ec1-4d83-95e4-dd157009c812 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:30.307470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a4882c73-4ec1-4d83-95e4-dd157009c812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:30.323623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 332/685] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 7 16:04:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:30.329967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8df969-4f40-4fe4-a35f-0ad7dd4d3cf5 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:30.461523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 354/686] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:30.470170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:30.488224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8df969-4f40-4fe4-a35f-0ad7dd4d3cf5 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:30.525986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eebbf-6cdb-4969-ab7d-a445b3a74722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:30.526855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eebbf-6cdb-4969-ab7d-a445b3a74722 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:30.527293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eebbf-6cdb-4969-ab7d-a445b3a74722 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:30.527727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968eebbf-6cdb-4969-ab7d-a445b3a74722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:30.602516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968eebbf-6cdb-4969-ab7d-a445b3a74722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:30.604233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968eebbf-6cdb-4969-ab7d-a445b3a74722 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:30.694002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:30.715950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 333/687] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:30.717537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4882c73-4ec1-4d83-95e4-dd157009c812 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:30.767808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eebbf-6cdb-4969-ab7d-a445b3a74722 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:30.788934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:30.829028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4882c73-4ec1-4d83-95e4-dd157009c812 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:30.877651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13a4b2-fbfe-4acc-a162-993dbffa1f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:30.878439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13a4b2-fbfe-4acc-a162-993dbffa1f40 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:30.878818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13a4b2-fbfe-4acc-a162-993dbffa1f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:30.879217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13a4b2-fbfe-4acc-a162-993dbffa1f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:30.891033 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13a4b2-fbfe-4acc-a162-993dbffa1f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:30.892752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13a4b2-fbfe-4acc-a162-993dbffa1f40 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-ba13a4b2-fbfe-4acc-a162-993dbffa1f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:30.909542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:30.910536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:30.911444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:30.911444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:30.911444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:30.913821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968eebbf-6cdb-4969-ab7d-a445b3a74722 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:30.915368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 355/688] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:30.919211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13a4b2-fbfe-4acc-a162-993dbffa1f40 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:31.004513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b721b-5ea5-48d5-967b-59067e639235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:31.005504 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b721b-5ea5-48d5-967b-59067e639235 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:31.006073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b721b-5ea5-48d5-967b-59067e639235 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:31.006525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b721b-5ea5-48d5-967b-59067e639235 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:31.031113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13a4b2-fbfe-4acc-a162-993dbffa1f40 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:31.033236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653b721b-5ea5-48d5-967b-59067e639235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-653b721b-5ea5-48d5-967b-59067e639235 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:31.038543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:31.039534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:31.040414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:31.049863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 334/689] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:31.051921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:31.057143 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b721b-5ea5-48d5-967b-59067e639235 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:31.070757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32fb75-7211-407c-b06f-ccef8dca3d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:31.071487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32fb75-7211-407c-b06f-ccef8dca3d83 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:31.071866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32fb75-7211-407c-b06f-ccef8dca3d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:31.072246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32fb75-7211-407c-b06f-ccef8dca3d83 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:31.072956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32fb75-7211-407c-b06f-ccef8dca3d83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:31.073331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32fb75-7211-407c-b06f-ccef8dca3d83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:31.074769 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32fb75-7211-407c-b06f-ccef8dca3d83 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:31.092159 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 356/690] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:31.094092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32fb75-7211-407c-b06f-ccef8dca3d83 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:31.173314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:31.199439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32fb75-7211-407c-b06f-ccef8dca3d83 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:31.237046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab04bbf-8ed9-4532-8d5b-d820b666be8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:31.237798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab04bbf-8ed9-4532-8d5b-d820b666be8d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:31.238152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab04bbf-8ed9-4532-8d5b-d820b666be8d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:31.238502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab04bbf-8ed9-4532-8d5b-d820b666be8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:31.245977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab04bbf-8ed9-4532-8d5b-d820b666be8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:31.247498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab04bbf-8ed9-4532-8d5b-d820b666be8d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-9ab04bbf-8ed9-4532-8d5b-d820b666be8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:31.259272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:31.260706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:31.261633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:31.261633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:31.261633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:31.261633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:31.261633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:31.261633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:31.261633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:31.261633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:31.264377 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 357/691] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:31.267354 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab04bbf-8ed9-4532-8d5b-d820b666be8d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:31.310294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:31.369752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b721b-5ea5-48d5-967b-59067e639235 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:31.389061 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab04bbf-8ed9-4532-8d5b-d820b666be8d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:31.442313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a68063-266a-4fc6-b2fd-bbc2b51ccbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:31.443105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a68063-266a-4fc6-b2fd-bbc2b51ccbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:31.443383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a68063-266a-4fc6-b2fd-bbc2b51ccbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:31.443657 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a68063-266a-4fc6-b2fd-bbc2b51ccbc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:31.444363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a68063-266a-4fc6-b2fd-bbc2b51ccbc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:31.445407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a68063-266a-4fc6-b2fd-bbc2b51ccbc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:31.446775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a68063-266a-4fc6-b2fd-bbc2b51ccbc3 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:31.457523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36748c1-474e-4173-bdb0-f8d3e480c5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:31.458133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36748c1-474e-4173-bdb0-f8d3e480c5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:31.458451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36748c1-474e-4173-bdb0-f8d3e480c5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:31.458722 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36748c1-474e-4173-bdb0-f8d3e480c5b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:31.459420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36748c1-474e-4173-bdb0-f8d3e480c5b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:31.459727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36748c1-474e-4173-bdb0-f8d3e480c5b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:31.461222 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36748c1-474e-4173-bdb0-f8d3e480c5b3 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:31.471694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 335/692] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:31.477254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a68063-266a-4fc6-b2fd-bbc2b51ccbc3 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:31.482059 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 358/693] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:31.487587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36748c1-474e-4173-bdb0-f8d3e480c5b3 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:31.626973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a68063-266a-4fc6-b2fd-bbc2b51ccbc3 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:31.640576 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:31.651285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:31.655543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36748c1-474e-4173-bdb0-f8d3e480c5b3 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:31.692485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6280c-2960-4863-9d08-2896643dc9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:31.693139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6280c-2960-4863-9d08-2896643dc9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:31.693421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6280c-2960-4863-9d08-2896643dc9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:31.693704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6280c-2960-4863-9d08-2896643dc9c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:31.703725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba104153-1fc1-496e-8997-c1deabdd5d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:31.704682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba104153-1fc1-496e-8997-c1deabdd5d85 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:31.705156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba104153-1fc1-496e-8997-c1deabdd5d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:31.705623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba104153-1fc1-496e-8997-c1deabdd5d85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:31.706531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba104153-1fc1-496e-8997-c1deabdd5d85 None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=69a8796195ff4dfb85655cdd91a0229d) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:31.707031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba104153-1fc1-496e-8997-c1deabdd5d85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:31.709873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba104153-1fc1-496e-8997-c1deabdd5d85 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:31.713859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6280c-2960-4863-9d08-2896643dc9c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:31.715310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6280c-2960-4863-9d08-2896643dc9c6 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:31.717170 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 359/694] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:31 2022] GET /identity/v3/role_assignments?user.id=69a8796195ff4dfb85655cdd91a0229d&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-3bb6280c-2960-4863-9d08-2896643dc9c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:31.735820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:31.736826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:31.737721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:31.737721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:31.737721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:31.737721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 336/695] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:31.742942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba104153-1fc1-496e-8997-c1deabdd5d85 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:31.873667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba104153-1fc1-496e-8997-c1deabdd5d85 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:31.913017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bb942-adfb-48db-bad6-e47b6f6157ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:31.913972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bb942-adfb-48db-bad6-e47b6f6157ab None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:31.914445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bb942-adfb-48db-bad6-e47b6f6157ab None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:31.914908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bb942-adfb-48db-bad6-e47b6f6157ab None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:31.915743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bb942-adfb-48db-bad6-e47b6f6157ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:31.916278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bb942-adfb-48db-bad6-e47b6f6157ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:31.917867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bb942-adfb-48db-bad6-e47b6f6157ab None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:31.932277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 360/696] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:31.941149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6280c-2960-4863-9d08-2896643dc9c6 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:32.075884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6280c-2960-4863-9d08-2896643dc9c6 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:32.116636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705dbcb5-af51-471a-aac6-0fc622ef7497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:32.117269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dbcb5-af51-471a-aac6-0fc622ef7497 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:32.117550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dbcb5-af51-471a-aac6-0fc622ef7497 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:32.117824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705dbcb5-af51-471a-aac6-0fc622ef7497 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:32.126324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705dbcb5-af51-471a-aac6-0fc622ef7497 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:32.127794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705dbcb5-af51-471a-aac6-0fc622ef7497 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-705dbcb5-af51-471a-aac6-0fc622ef7497 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:32.139831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:32.140867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:32.141864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:32.141864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:32.141864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:32.141864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:32.141864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:32.141864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:32.141864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:32.141864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:32.141864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:32.141864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:32.141864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 337/697] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 7 16:04:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:32.146950 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bb942-adfb-48db-bad6-e47b6f6157ab None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:32.183846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:32.198024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5e2f-2df4-4c1c-b3cd-2fb04cb4ea78 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:32.198336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5e2f-2df4-4c1c-b3cd-2fb04cb4ea78 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:32.198572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5e2f-2df4-4c1c-b3cd-2fb04cb4ea78 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:32.200337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 338/698] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:32.208486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98437dfc-0af2-40e6-b99e-21e426a53d01 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:32.208486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98437dfc-0af2-40e6-b99e-21e426a53d01 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:32.208780 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98437dfc-0af2-40e6-b99e-21e426a53d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:32.232256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-98437dfc-0af2-40e6-b99e-21e426a53d01 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:32.234046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-98437dfc-0af2-40e6-b99e-21e426a53d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:32.251730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:32.294224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bb942-adfb-48db-bad6-e47b6f6157ab None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:32.328125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 339/699] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:32.342015 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc93851-1bca-4560-861b-d62d8341c6fe None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:32.342507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc93851-1bca-4560-861b-d62d8341c6fe None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:32.343584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc93851-1bca-4560-861b-d62d8341c6fe None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:32.345471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 340/700] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:04:32.345949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828748c-c7bf-45c5-8507-f2aebdb7a32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:32.346570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828748c-c7bf-45c5-8507-f2aebdb7a32f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:32.346854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828748c-c7bf-45c5-8507-f2aebdb7a32f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:32.347129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828748c-c7bf-45c5-8507-f2aebdb7a32f None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:32.347900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828748c-c7bf-45c5-8507-f2aebdb7a32f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:32.348211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828748c-c7bf-45c5-8507-f2aebdb7a32f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:32.349649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828748c-c7bf-45c5-8507-f2aebdb7a32f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:32.356649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797572-b1a4-4f97-a09c-8ae59b456547 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:32.357188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797572-b1a4-4f97-a09c-8ae59b456547 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:32.357723 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae797572-b1a4-4f97-a09c-8ae59b456547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:32.377998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ae797572-b1a4-4f97-a09c-8ae59b456547 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:32.379632 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ae797572-b1a4-4f97-a09c-8ae59b456547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:32.382304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 361/701] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 7 16:04:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:32.388402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828748c-c7bf-45c5-8507-f2aebdb7a32f None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:32.506443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 341/702] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:32.517705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:32.536950 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828748c-c7bf-45c5-8507-f2aebdb7a32f None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:32.608477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88460a4-a21e-4bc0-9052-e4ae178fa4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:32.609095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88460a4-a21e-4bc0-9052-e4ae178fa4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:32.609371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88460a4-a21e-4bc0-9052-e4ae178fa4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:32.609640 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88460a4-a21e-4bc0-9052-e4ae178fa4f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:32.689412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88460a4-a21e-4bc0-9052-e4ae178fa4f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:32.690870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88460a4-a21e-4bc0-9052-e4ae178fa4f5 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:32.773917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:32.776818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae797572-b1a4-4f97-a09c-8ae59b456547 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:32.795808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 362/703] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 16:04:32.799806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88460a4-a21e-4bc0-9052-e4ae178fa4f5 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:32.892650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:32.900163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88460a4-a21e-4bc0-9052-e4ae178fa4f5 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:32.922308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae797572-b1a4-4f97-a09c-8ae59b456547 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:32.938804 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c87ff-8662-4059-a365-8a2256cb843e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:32.939445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c87ff-8662-4059-a365-8a2256cb843e None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:32.939723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c87ff-8662-4059-a365-8a2256cb843e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:32.940029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c87ff-8662-4059-a365-8a2256cb843e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:32.950799 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c87ff-8662-4059-a365-8a2256cb843e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:32.952493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c87ff-8662-4059-a365-8a2256cb843e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:32.966370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c7b5e-bcaa-40ba-a5aa-f70183fb86da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:32.967007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c7b5e-bcaa-40ba-a5aa-f70183fb86da None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:32.967295 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c7b5e-bcaa-40ba-a5aa-f70183fb86da None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:32.967572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c7b5e-bcaa-40ba-a5aa-f70183fb86da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-4f3c87ff-8662-4059-a365-8a2256cb843e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:32.968632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:32.969538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:32.971007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:32.971007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:32.971007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:32.971007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 363/704] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:32.971007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c87ff-8662-4059-a365-8a2256cb843e None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:32.986217 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c7b5e-bcaa-40ba-a5aa-f70183fb86da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:32.987951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c7b5e-bcaa-40ba-a5aa-f70183fb86da None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:33.011273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-835c7b5e-bcaa-40ba-a5aa-f70183fb86da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:33.022496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:33.023704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:33.024892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:33.024892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:33.024892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:33.026996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 342/705] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:33.028636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c7b5e-bcaa-40ba-a5aa-f70183fb86da None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:33.087835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c87ff-8662-4059-a365-8a2256cb843e None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:33.126361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23727d77-f156-45a9-8bf1-489956342911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:33.127199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727d77-f156-45a9-8bf1-489956342911 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:33.127571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727d77-f156-45a9-8bf1-489956342911 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:33.128029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23727d77-f156-45a9-8bf1-489956342911 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:33.128725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23727d77-f156-45a9-8bf1-489956342911 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:33.130238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23727d77-f156-45a9-8bf1-489956342911 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-23727d77-f156-45a9-8bf1-489956342911 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:33.141356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:33.142643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 16:04:33.143683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:33.147873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 364/706] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Sep 7 16:04:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:33.147873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23727d77-f156-45a9-8bf1-489956342911 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:33.170902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c7b5e-bcaa-40ba-a5aa-f70183fb86da None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:33.215003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:33.223128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30c292-7ec9-46e4-b082-0ca8229d2b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:33.224165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30c292-7ec9-46e4-b082-0ca8229d2b8f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:33.224793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30c292-7ec9-46e4-b082-0ca8229d2b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:33.225265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30c292-7ec9-46e4-b082-0ca8229d2b8f None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:33.226126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30c292-7ec9-46e4-b082-0ca8229d2b8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:33.226636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30c292-7ec9-46e4-b082-0ca8229d2b8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:33.228488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30c292-7ec9-46e4-b082-0ca8229d2b8f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:33.242815 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 343/707] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:33.243952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30c292-7ec9-46e4-b082-0ca8229d2b8f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:33.273053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:33.286760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23727d77-f156-45a9-8bf1-489956342911 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:33.321705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2b043-d5d0-4728-8b03-886d9b54b9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:33.322346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2b043-d5d0-4728-8b03-886d9b54b9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:33.322571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2b043-d5d0-4728-8b03-886d9b54b9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:33.322837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a2b043-d5d0-4728-8b03-886d9b54b9f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:33.323915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2b043-d5d0-4728-8b03-886d9b54b9f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:33.324218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2b043-d5d0-4728-8b03-886d9b54b9f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:33.325926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a2b043-d5d0-4728-8b03-886d9b54b9f7 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:33.339937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 365/708] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:33.339937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2b043-d5d0-4728-8b03-886d9b54b9f7 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:33.352097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30c292-7ec9-46e4-b082-0ca8229d2b8f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:33.387607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166391fc-7bad-4726-9625-bf663948de3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:33.388248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166391fc-7bad-4726-9625-bf663948de3b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:33.388530 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166391fc-7bad-4726-9625-bf663948de3b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:33.388807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166391fc-7bad-4726-9625-bf663948de3b None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:33.389438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166391fc-7bad-4726-9625-bf663948de3b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:33.390277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166391fc-7bad-4726-9625-bf663948de3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:33.391640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166391fc-7bad-4726-9625-bf663948de3b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:33.395986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 344/709] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 7 16:04:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:33.397102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca686-1da8-4069-b85a-e8b90a24adc8 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:33.397386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca686-1da8-4069-b85a-e8b90a24adc8 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:33.397664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ca686-1da8-4069-b85a-e8b90a24adc8 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:33.403023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 345/710] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:33.403791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ca686-1da8-4069-b85a-e8b90a24adc8 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:33.452814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a2b043-d5d0-4728-8b03-886d9b54b9f7 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:33.464253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:33.487729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a686969-e69a-4a56-8335-354f4744e5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:33.488391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a686969-e69a-4a56-8335-354f4744e5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:33.488683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a686969-e69a-4a56-8335-354f4744e5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:33.488959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a686969-e69a-4a56-8335-354f4744e5c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:33.496320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a686969-e69a-4a56-8335-354f4744e5c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:33.497790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a686969-e69a-4a56-8335-354f4744e5c0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-0a686969-e69a-4a56-8335-354f4744e5c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:33.508878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:33.514511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:33.516337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:33.516337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:33.516337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:33.516337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:33.516337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:33.516337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:33.516337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:33.516337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:33.516337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 366/711] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:33.516337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a686969-e69a-4a56-8335-354f4744e5c0 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:33.525374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ca686-1da8-4069-b85a-e8b90a24adc8 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:33.531316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:33.562353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98afd8-601b-48f1-860c-6392e015eef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:33.563050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98afd8-601b-48f1-860c-6392e015eef7 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:33.563395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98afd8-601b-48f1-860c-6392e015eef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:33.563728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b98afd8-601b-48f1-860c-6392e015eef7 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:33.564473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98afd8-601b-48f1-860c-6392e015eef7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:33.565946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b98afd8-601b-48f1-860c-6392e015eef7 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:33.615260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a686969-e69a-4a56-8335-354f4744e5c0 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:33.649130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c700f7f-bd0c-460b-8d30-00c81b60fbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:33.651684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c700f7f-bd0c-460b-8d30-00c81b60fbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:33.651684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c700f7f-bd0c-460b-8d30-00c81b60fbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:33.651684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c700f7f-bd0c-460b-8d30-00c81b60fbd2 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:33.651684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c700f7f-bd0c-460b-8d30-00c81b60fbd2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:33.651684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c700f7f-bd0c-460b-8d30-00c81b60fbd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:33.652645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c700f7f-bd0c-460b-8d30-00c81b60fbd2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:33.658644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 367/712] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 7 16:04:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:33.659737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296863c6-dcc5-4a08-8238-7b82389f79be None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:33.660042 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296863c6-dcc5-4a08-8238-7b82389f79be None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:33.660312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296863c6-dcc5-4a08-8238-7b82389f79be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:33.690681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-296863c6-dcc5-4a08-8238-7b82389f79be None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:33.690910 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 346/713] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:04:33 2022] POST /identity/v3/users => generated 254 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:04:33.691826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98afd8-601b-48f1-860c-6392e015eef7 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:33.692178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-296863c6-dcc5-4a08-8238-7b82389f79be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:33.784346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 368/714] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:33.887919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b98afd8-601b-48f1-860c-6392e015eef7 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:33.900192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:33.927407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edd5ad-deb3-4d38-a573-49321c1699a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:33.928068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edd5ad-deb3-4d38-a573-49321c1699a5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:33.928344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edd5ad-deb3-4d38-a573-49321c1699a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:33.928616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49edd5ad-deb3-4d38-a573-49321c1699a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:33.929322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edd5ad-deb3-4d38-a573-49321c1699a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:33.929623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edd5ad-deb3-4d38-a573-49321c1699a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:33.931429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49edd5ad-deb3-4d38-a573-49321c1699a5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:33.948040 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 347/715] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:33.954452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296863c6-dcc5-4a08-8238-7b82389f79be None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:33.971537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:34.070086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296863c6-dcc5-4a08-8238-7b82389f79be None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:34.105487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76627a6-2309-4cae-a030-2f6a8141283b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:34.106390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76627a6-2309-4cae-a030-2f6a8141283b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:34.106843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76627a6-2309-4cae-a030-2f6a8141283b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:34.107282 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76627a6-2309-4cae-a030-2f6a8141283b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:34.108226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76627a6-2309-4cae-a030-2f6a8141283b None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=45061db68b1e40f59935cc9a46d2ff10) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:34.108740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76627a6-2309-4cae-a030-2f6a8141283b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:34.110331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76627a6-2309-4cae-a030-2f6a8141283b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:34.116259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 369/716] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:33 2022] GET /identity/v3/role_assignments?user.id=45061db68b1e40f59935cc9a46d2ff10&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:34.132862 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edd5ad-deb3-4d38-a573-49321c1699a5 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:34.246399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49edd5ad-deb3-4d38-a573-49321c1699a5 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:34.282958 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51312046-85b0-4416-9cfe-c56e6a97ad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:34.283983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51312046-85b0-4416-9cfe-c56e6a97ad72 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:34.284438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51312046-85b0-4416-9cfe-c56e6a97ad72 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:34.284884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51312046-85b0-4416-9cfe-c56e6a97ad72 None admin] PATH_INFO: `/v3/services/network` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:34.285694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51312046-85b0-4416-9cfe-c56e6a97ad72 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:34.287481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51312046-85b0-4416-9cfe-c56e6a97ad72 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-51312046-85b0-4416-9cfe-c56e6a97ad72 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:34.300672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:34.301714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 16:04:34.302536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:34.305018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 348/717] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:34 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:34.313904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76627a6-2309-4cae-a030-2f6a8141283b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:34.400128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:34.419129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76627a6-2309-4cae-a030-2f6a8141283b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:34.459071 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1feaec1-17b8-4a65-813e-890bea60906a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:34.459679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feaec1-17b8-4a65-813e-890bea60906a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:34.459986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feaec1-17b8-4a65-813e-890bea60906a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:34.460264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feaec1-17b8-4a65-813e-890bea60906a None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:34.460893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1feaec1-17b8-4a65-813e-890bea60906a None admin] RBAC: Adding query filter params (name=network) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:34.461247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1feaec1-17b8-4a65-813e-890bea60906a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:34.462659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1feaec1-17b8-4a65-813e-890bea60906a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:34.467443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 370/718] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:04:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:34.473002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51312046-85b0-4416-9cfe-c56e6a97ad72 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:34.513154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:34.597737 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51312046-85b0-4416-9cfe-c56e6a97ad72 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:34.619527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb7894-3b58-4b13-bb1f-83c65a17047c None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:34.619844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb7894-3b58-4b13-bb1f-83c65a17047c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:34.620125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb7894-3b58-4b13-bb1f-83c65a17047c None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:34.622750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 371/719] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:04:34.626798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d2a47-bf11-49ac-8e12-43b89798e3a2 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:34.627085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d2a47-bf11-49ac-8e12-43b89798e3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:34.627362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d2a47-bf11-49ac-8e12-43b89798e3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:34.634558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac1cfa-abca-46e9-b3c6-326049f865c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:34.636045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac1cfa-abca-46e9-b3c6-326049f865c3 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:34.636327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac1cfa-abca-46e9-b3c6-326049f865c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:34.636600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac1cfa-abca-46e9-b3c6-326049f865c3 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:34.637273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac1cfa-abca-46e9-b3c6-326049f865c3 None admin] RBAC: Adding query filter params (type=network) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:34.637747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac1cfa-abca-46e9-b3c6-326049f865c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:34.639708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac1cfa-abca-46e9-b3c6-326049f865c3 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:34.647204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 349/720] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:04:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:34.649000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-4a4d2a47-bf11-49ac-8e12-43b89798e3a2 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:34.652267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4a4d2a47-bf11-49ac-8e12-43b89798e3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:34.766319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 372/721] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:35.172293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac1cfa-abca-46e9-b3c6-326049f865c3 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:35.292601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:35.300092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac1cfa-abca-46e9-b3c6-326049f865c3 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:35.350922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be4d94-5c2c-4801-96c6-eeeeb048d326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:35.351805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be4d94-5c2c-4801-96c6-eeeeb048d326 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:35.351916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be4d94-5c2c-4801-96c6-eeeeb048d326 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:35.352121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be4d94-5c2c-4801-96c6-eeeeb048d326 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:35.367661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be4d94-5c2c-4801-96c6-eeeeb048d326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-a7be4d94-5c2c-4801-96c6-eeeeb048d326 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:35.372416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:35.373686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:35.375162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 350/722] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:35.379919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d2a47-bf11-49ac-8e12-43b89798e3a2 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:35.480282 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:35.524240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d2a47-bf11-49ac-8e12-43b89798e3a2 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:35.566021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ba5f4-f61f-4d70-8a8e-5ebc69087346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:35.566635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ba5f4-f61f-4d70-8a8e-5ebc69087346 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:35.566963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ba5f4-f61f-4d70-8a8e-5ebc69087346 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:35.567249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253ba5f4-f61f-4d70-8a8e-5ebc69087346 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:35.567953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ba5f4-f61f-4d70-8a8e-5ebc69087346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:35.568243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ba5f4-f61f-4d70-8a8e-5ebc69087346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:35.569605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253ba5f4-f61f-4d70-8a8e-5ebc69087346 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:35.578572 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 373/723] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:35.584306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be4d94-5c2c-4801-96c6-eeeeb048d326 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:35.732322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be4d94-5c2c-4801-96c6-eeeeb048d326 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:35.785812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec013650-89ca-4e34-b9e3-7123a9293d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:35.786444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec013650-89ca-4e34-b9e3-7123a9293d4f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:35.786722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec013650-89ca-4e34-b9e3-7123a9293d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:35.786991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec013650-89ca-4e34-b9e3-7123a9293d4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:35.803477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec013650-89ca-4e34-b9e3-7123a9293d4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:35.804948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec013650-89ca-4e34-b9e3-7123a9293d4f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-ec013650-89ca-4e34-b9e3-7123a9293d4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:35.820125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:35.821189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:35.822603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:35.822603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:35.822603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:35.822603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 351/724] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:35.829014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ba5f4-f61f-4d70-8a8e-5ebc69087346 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:35.971223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253ba5f4-f61f-4d70-8a8e-5ebc69087346 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:36.016245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:36.024409 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdb2b8-0cc8-4eb9-81e0-bef2929fec90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:36.025033 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdb2b8-0cc8-4eb9-81e0-bef2929fec90 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:36.025315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdb2b8-0cc8-4eb9-81e0-bef2929fec90 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:36.025590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bdb2b8-0cc8-4eb9-81e0-bef2929fec90 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:36.026238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdb2b8-0cc8-4eb9-81e0-bef2929fec90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:36.026542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdb2b8-0cc8-4eb9-81e0-bef2929fec90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:36.027977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bdb2b8-0cc8-4eb9-81e0-bef2929fec90 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:36.040659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 374/725] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:36.049008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec013650-89ca-4e34-b9e3-7123a9293d4f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:36.079629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:36.200395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec013650-89ca-4e34-b9e3-7123a9293d4f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:36.241868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e50233-b7dc-47e6-8d01-72952c50a0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:36.242886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e50233-b7dc-47e6-8d01-72952c50a0ff None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:36.243373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e50233-b7dc-47e6-8d01-72952c50a0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:36.243898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e50233-b7dc-47e6-8d01-72952c50a0ff None admin] PATH_INFO: `/v3/users/swift` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:36.255557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e50233-b7dc-47e6-8d01-72952c50a0ff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:36.257298 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e50233-b7dc-47e6-8d01-72952c50a0ff None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-e1e50233-b7dc-47e6-8d01-72952c50a0ff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:36.271585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:36.272624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:36.273607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:36.273607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:36.273607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:36.273607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:36.273607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:36.273607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:36.273607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:36.273607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:36.273607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:36.273607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:36.276170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 352/726] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:36.282510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdb2b8-0cc8-4eb9-81e0-bef2929fec90 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:36.410294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bdb2b8-0cc8-4eb9-81e0-bef2929fec90 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:36.447448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22370e5a-3229-4d29-b0e4-979f03c36bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:36.448471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22370e5a-3229-4d29-b0e4-979f03c36bba None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:36.448982 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22370e5a-3229-4d29-b0e4-979f03c36bba None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:36.449460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22370e5a-3229-4d29-b0e4-979f03c36bba None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:36.450381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22370e5a-3229-4d29-b0e4-979f03c36bba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:36.450888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22370e5a-3229-4d29-b0e4-979f03c36bba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:36.452629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22370e5a-3229-4d29-b0e4-979f03c36bba None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:36.470674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 375/727] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 7 16:04:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:36.476639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e50233-b7dc-47e6-8d01-72952c50a0ff None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:36.556663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:36.591022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e50233-b7dc-47e6-8d01-72952c50a0ff None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:36.629591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fb28b-eff3-4fc9-bf16-3ada627335fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:36.630557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb28b-eff3-4fc9-bf16-3ada627335fb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:36.631067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb28b-eff3-4fc9-bf16-3ada627335fb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:36.631542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fb28b-eff3-4fc9-bf16-3ada627335fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:36.685731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5cf93-dcf2-46bd-a246-e936a3c3c69c None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:36.686315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5cf93-dcf2-46bd-a246-e936a3c3c69c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:36.686315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf5cf93-dcf2-46bd-a246-e936a3c3c69c None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:36.688511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 376/728] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:36.694136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3eba9-eacc-449b-b8e5-49a8de9f6599 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:36.694457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3eba9-eacc-449b-b8e5-49a8de9f6599 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:36.694714 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3eba9-eacc-449b-b8e5-49a8de9f6599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:36.713937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-c4c3eba9-eacc-449b-b8e5-49a8de9f6599 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:36.715139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c4c3eba9-eacc-449b-b8e5-49a8de9f6599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:36.719735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fb28b-eff3-4fc9-bf16-3ada627335fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:36.721484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fb28b-eff3-4fc9-bf16-3ada627335fb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:36.761737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:36.786419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 377/729] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:36.807793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:36.841032 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 353/730] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 16:04:36.847225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3eba9-eacc-449b-b8e5-49a8de9f6599 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:36.881064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02240761-19b9-4c6c-b778-409d39b71eef None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:36.881476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02240761-19b9-4c6c-b778-409d39b71eef None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:36.881756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02240761-19b9-4c6c-b778-409d39b71eef None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:36.883481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 354/731] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:36.894570 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4654-fcfb-4417-8fb5-23eba299b58a None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:36.894850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4654-fcfb-4417-8fb5-23eba299b58a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:36.895125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda4654-fcfb-4417-8fb5-23eba299b58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:36.924024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-efda4654-fcfb-4417-8fb5-23eba299b58a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:36.925233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-efda4654-fcfb-4417-8fb5-23eba299b58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:37.032262 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 355/732] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:37.046305 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3eba9-eacc-449b-b8e5-49a8de9f6599 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:37.072687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bbee0-322e-4616-ae90-c000166d302b None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.072897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bbee0-322e-4616-ae90-c000166d302b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.073604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bbee0-322e-4616-ae90-c000166d302b None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.077059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 356/733] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:37.084897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d8251-d82c-4c6d-98a8-ffffefb5f737 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.085193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d8251-d82c-4c6d-98a8-ffffefb5f737 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.085476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d8251-d82c-4c6d-98a8-ffffefb5f737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.107881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6b402-5094-45b9-8ae7-6936155ba398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:37.109093 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6b402-5094-45b9-8ae7-6936155ba398 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.109374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6b402-5094-45b9-8ae7-6936155ba398 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.109651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6b402-5094-45b9-8ae7-6936155ba398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.116199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0b4d8251-d82c-4c6d-98a8-ffffefb5f737 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:37.117609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0b4d8251-d82c-4c6d-98a8-ffffefb5f737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:37.122639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6b402-5094-45b9-8ae7-6936155ba398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:37.124251 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6b402-5094-45b9-8ae7-6936155ba398 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-b6a6b402-5094-45b9-8ae7-6936155ba398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:37.145201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:37.146202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:37.147111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:37.147111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:37.147111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:37.147111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 378/734] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:37.147886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6b402-5094-45b9-8ae7-6936155ba398 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:37.186964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:37.215795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 357/735] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 16:04:37.215795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d8251-d82c-4c6d-98a8-ffffefb5f737 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:37.248098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:37.324775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6b402-5094-45b9-8ae7-6936155ba398 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:37.343964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d8251-d82c-4c6d-98a8-ffffefb5f737 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:37.362407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398ef61-b174-49f3-965b-53bbb6e67f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:37.363484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398ef61-b174-49f3-965b-53bbb6e67f68 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.364051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398ef61-b174-49f3-965b-53bbb6e67f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.364338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398ef61-b174-49f3-965b-53bbb6e67f68 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.364994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398ef61-b174-49f3-965b-53bbb6e67f68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:37.367053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398ef61-b174-49f3-965b-53bbb6e67f68 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:37.384922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373217c6-83dd-44a9-acd6-78dc342c8543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:37.385647 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373217c6-83dd-44a9-acd6-78dc342c8543 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.386056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373217c6-83dd-44a9-acd6-78dc342c8543 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.386336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373217c6-83dd-44a9-acd6-78dc342c8543 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.387037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373217c6-83dd-44a9-acd6-78dc342c8543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:37.387331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373217c6-83dd-44a9-acd6-78dc342c8543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:37.388740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373217c6-83dd-44a9-acd6-78dc342c8543 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:37.401912 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 358/736] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:37.404572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373217c6-83dd-44a9-acd6-78dc342c8543 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:37.471825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 379/737] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:04:37 2022] POST /identity/v3/services => generated 251 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:04:37.471825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398ef61-b174-49f3-965b-53bbb6e67f68 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:37.528216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:37.534276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373217c6-83dd-44a9-acd6-78dc342c8543 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:37.586756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ca0bc-79d9-4257-b704-479ea0010a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:37.587321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ca0bc-79d9-4257-b704-479ea0010a36 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.587605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ca0bc-79d9-4257-b704-479ea0010a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.587918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ca0bc-79d9-4257-b704-479ea0010a36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.594707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:37.601757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ca0bc-79d9-4257-b704-479ea0010a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:37.606387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398ef61-b174-49f3-965b-53bbb6e67f68 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-915ca0bc-79d9-4257-b704-479ea0010a36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:37.606784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:37.610640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 359/738] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:37.617357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fef42-820e-44db-bb13-44c3f582d36e None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.618139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fef42-820e-44db-bb13-44c3f582d36e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.618139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fef42-820e-44db-bb13-44c3f582d36e None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.618139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 360/739] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:37.618139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fef42-820e-44db-bb13-44c3f582d36e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:37.655432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d7c0d-30a2-4377-b08d-86146d2705b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:37.656107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d7c0d-30a2-4377-b08d-86146d2705b3 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.656384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d7c0d-30a2-4377-b08d-86146d2705b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.656655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d7c0d-30a2-4377-b08d-86146d2705b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.664381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d7c0d-30a2-4377-b08d-86146d2705b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:37.665858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d7c0d-30a2-4377-b08d-86146d2705b3 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-749d7c0d-30a2-4377-b08d-86146d2705b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:37.679655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:37.680941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:37.681843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:37.681843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:37.681843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:37.681843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:37.681843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:37.681843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:37.681843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:37.681843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:37.684596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 380/740] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:37.685716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d7c0d-30a2-4377-b08d-86146d2705b3 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:37.800156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d7c0d-30a2-4377-b08d-86146d2705b3 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:37.838813 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef61e3d-f2ec-42f4-9008-069c74eb31a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:37.838998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fef42-820e-44db-bb13-44c3f582d36e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:37.839402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61e3d-f2ec-42f4-9008-069c74eb31a8 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.839639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61e3d-f2ec-42f4-9008-069c74eb31a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.839897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef61e3d-f2ec-42f4-9008-069c74eb31a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.840549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef61e3d-f2ec-42f4-9008-069c74eb31a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:37.840846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef61e3d-f2ec-42f4-9008-069c74eb31a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:37.842204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef61e3d-f2ec-42f4-9008-069c74eb31a8 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:37.854909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 381/741] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:37.856185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e7479-e9aa-4bce-86e1-edf0db8100fc None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.856474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e7479-e9aa-4bce-86e1-edf0db8100fc None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.856746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e7479-e9aa-4bce-86e1-edf0db8100fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.876282 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-dd1e7479-e9aa-4bce-86e1-edf0db8100fc None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:37.876579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaead14-3b91-4ce6-9556-eee8a5519fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:37.877310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaead14-3b91-4ce6-9556-eee8a5519fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.877476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-dd1e7479-e9aa-4bce-86e1-edf0db8100fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:37.877773 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaead14-3b91-4ce6-9556-eee8a5519fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.878180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaead14-3b91-4ce6-9556-eee8a5519fd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.885163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:37.891182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaead14-3b91-4ce6-9556-eee8a5519fd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-bcaead14-3b91-4ce6-9556-eee8a5519fd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:37.896130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:37.897063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 361/742] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:37.900779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaead14-3b91-4ce6-9556-eee8a5519fd7 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:37.908738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:37.953371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 382/743] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 16:04:37.954526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e7479-e9aa-4bce-86e1-edf0db8100fc None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:38.020000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaead14-3b91-4ce6-9556-eee8a5519fd7 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:38.056987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59718d-aab4-4b43-b2d8-6beefd113ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:38.057892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59718d-aab4-4b43-b2d8-6beefd113ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:38.058343 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59718d-aab4-4b43-b2d8-6beefd113ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:38.058713 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59718d-aab4-4b43-b2d8-6beefd113ab1 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:38.059521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59718d-aab4-4b43-b2d8-6beefd113ab1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:38.059958 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59718d-aab4-4b43-b2d8-6beefd113ab1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:38.062191 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59718d-aab4-4b43-b2d8-6beefd113ab1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:38.077331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 362/744] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:38.078203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59718d-aab4-4b43-b2d8-6beefd113ab1 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:38.107002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e7479-e9aa-4bce-86e1-edf0db8100fc None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:38.152859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93498f-953c-4258-b5b3-235232f4a3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:38.153644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93498f-953c-4258-b5b3-235232f4a3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:38.154044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93498f-953c-4258-b5b3-235232f4a3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:38.154414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93498f-953c-4258-b5b3-235232f4a3b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:38.156775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:38.167304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93498f-953c-4258-b5b3-235232f4a3b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:38.169056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93498f-953c-4258-b5b3-235232f4a3b0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:38.171618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-5d93498f-953c-4258-b5b3-235232f4a3b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:38.187243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:38.188275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:38.189190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:38.189190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:38.189190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:38.189190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 383/745] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:38.190100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93498f-953c-4258-b5b3-235232f4a3b0 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:38.198593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59718d-aab4-4b43-b2d8-6beefd113ab1 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:38.234972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87876aaf-1b82-418b-9c77-c3fbbd51447e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:38.235975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87876aaf-1b82-418b-9c77-c3fbbd51447e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:38.236464 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87876aaf-1b82-418b-9c77-c3fbbd51447e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:38.236940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87876aaf-1b82-418b-9c77-c3fbbd51447e None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:38.237851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87876aaf-1b82-418b-9c77-c3fbbd51447e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:38.238363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87876aaf-1b82-418b-9c77-c3fbbd51447e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:38.239977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87876aaf-1b82-418b-9c77-c3fbbd51447e None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:38.249596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 363/746] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:38.250678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87876aaf-1b82-418b-9c77-c3fbbd51447e None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:38.302210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93498f-953c-4258-b5b3-235232f4a3b0 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:38.338775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c844a91-0b4b-4e8e-b4c4-509ea1bff6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:38.339378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c844a91-0b4b-4e8e-b4c4-509ea1bff6bd None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:38.339656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c844a91-0b4b-4e8e-b4c4-509ea1bff6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:38.340075 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c844a91-0b4b-4e8e-b4c4-509ea1bff6bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:38.340909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c844a91-0b4b-4e8e-b4c4-509ea1bff6bd None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=69a8796195ff4dfb85655cdd91a0229d) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:38.341219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c844a91-0b4b-4e8e-b4c4-509ea1bff6bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:38.343172 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c844a91-0b4b-4e8e-b4c4-509ea1bff6bd None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:38.359890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 384/747] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:38 2022] GET /identity/v3/role_assignments?user.id=69a8796195ff4dfb85655cdd91a0229d&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:38.359890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c844a91-0b4b-4e8e-b4c4-509ea1bff6bd None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:38.372521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87876aaf-1b82-418b-9c77-c3fbbd51447e None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:38.407165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:38.416063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82273ce-24c8-43bc-b723-1ca9d1c44e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:38.416926 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82273ce-24c8-43bc-b723-1ca9d1c44e6b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:38.417374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82273ce-24c8-43bc-b723-1ca9d1c44e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:38.417976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82273ce-24c8-43bc-b723-1ca9d1c44e6b None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:38.418730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82273ce-24c8-43bc-b723-1ca9d1c44e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:38.419120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82273ce-24c8-43bc-b723-1ca9d1c44e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:38.421678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82273ce-24c8-43bc-b723-1ca9d1c44e6b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:38.440407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 364/748] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:38.441762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:38.442430 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82273ce-24c8-43bc-b723-1ca9d1c44e6b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:38.496762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c844a91-0b4b-4e8e-b4c4-509ea1bff6bd None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:38.533287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f678b-8883-449c-8e1a-714bad9b56b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:38.533928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f678b-8883-449c-8e1a-714bad9b56b8 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:38.534210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f678b-8883-449c-8e1a-714bad9b56b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:38.534485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f678b-8883-449c-8e1a-714bad9b56b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:38.548165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f678b-8883-449c-8e1a-714bad9b56b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:38.549524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f678b-8883-449c-8e1a-714bad9b56b8 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:38.559617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82273ce-24c8-43bc-b723-1ca9d1c44e6b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-228f678b-8883-449c-8e1a-714bad9b56b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:38.564414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:38.565352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:38.566296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:38.566296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:38.566296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:38.570218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 385/749] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:38.571070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f678b-8883-449c-8e1a-714bad9b56b8 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:38.606200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6c6da-3d60-474e-a3d5-1ebde5860db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:38.607249 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6c6da-3d60-474e-a3d5-1ebde5860db2 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:38.607721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6c6da-3d60-474e-a3d5-1ebde5860db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:38.608941 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6c6da-3d60-474e-a3d5-1ebde5860db2 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:38.609791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6c6da-3d60-474e-a3d5-1ebde5860db2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:38.611460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6c6da-3d60-474e-a3d5-1ebde5860db2 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:38.681307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f678b-8883-449c-8e1a-714bad9b56b8 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:38.692415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:38.708175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 365/750] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:04:38 2022] POST /identity/v3/services => generated 240 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:04:38.716694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6c6da-3d60-474e-a3d5-1ebde5860db2 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:38.740330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2584e6-9021-4543-be55-1c7af28fd7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:38.741287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2584e6-9021-4543-be55-1c7af28fd7ae None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:38.741730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2584e6-9021-4543-be55-1c7af28fd7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:38.742938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2584e6-9021-4543-be55-1c7af28fd7ae None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:38.755499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2584e6-9021-4543-be55-1c7af28fd7ae None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:38.757574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2584e6-9021-4543-be55-1c7af28fd7ae None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-aa2584e6-9021-4543-be55-1c7af28fd7ae None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:38.773217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:38.774468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:38.775694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:38.775694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:38.775694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:38.775694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:38.775694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:38.775694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:38.775694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:38.775694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:38.775694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:04:38.775694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:38.778276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 386/751] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 7 16:04:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:38.784414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2584e6-9021-4543-be55-1c7af28fd7ae None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:38.842320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:38.945312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6c6da-3d60-474e-a3d5-1ebde5860db2 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:38.988459 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534d54c-1ab6-417e-a03b-2b54bb460cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:38.989458 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534d54c-1ab6-417e-a03b-2b54bb460cfd None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:38.989741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534d54c-1ab6-417e-a03b-2b54bb460cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:38.990017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0534d54c-1ab6-417e-a03b-2b54bb460cfd None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:38.990668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534d54c-1ab6-417e-a03b-2b54bb460cfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:38.990968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534d54c-1ab6-417e-a03b-2b54bb460cfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:38.992383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0534d54c-1ab6-417e-a03b-2b54bb460cfd None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:39.005696 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2584e6-9021-4543-be55-1c7af28fd7ae None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:39.012778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 366/752] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:39.013917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534d54c-1ab6-417e-a03b-2b54bb460cfd None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:39.136014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee7cf5-8c5e-41a4-a2f0-f9a8d70ed830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:39.136628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee7cf5-8c5e-41a4-a2f0-f9a8d70ed830 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:39.136905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee7cf5-8c5e-41a4-a2f0-f9a8d70ed830 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:39.137174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee7cf5-8c5e-41a4-a2f0-f9a8d70ed830 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:39.137900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee7cf5-8c5e-41a4-a2f0-f9a8d70ed830 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:39.138205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee7cf5-8c5e-41a4-a2f0-f9a8d70ed830 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:39.139661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee7cf5-8c5e-41a4-a2f0-f9a8d70ed830 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:39.179143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 387/753] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 7 16:04:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:39.183025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0534d54c-1ab6-417e-a03b-2b54bb460cfd None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:39.185217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee7cf5-8c5e-41a4-a2f0-f9a8d70ed830 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:39.208192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:39.214079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:39.289110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b334a-6d94-4ca2-8d42-412e63682f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:39.289772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b334a-6d94-4ca2-8d42-412e63682f78 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:39.290055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b334a-6d94-4ca2-8d42-412e63682f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:39.290335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b334a-6d94-4ca2-8d42-412e63682f78 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:39.298236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee7cf5-8c5e-41a4-a2f0-f9a8d70ed830 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:39.317132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b334a-6d94-4ca2-8d42-412e63682f78 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:39.318577 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b334a-6d94-4ca2-8d42-412e63682f78 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-fc1b334a-6d94-4ca2-8d42-412e63682f78 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:39.338599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:39.339809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:39.340977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:39.340977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:39.340977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:39.340977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:39.340977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:39.340977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:39.340977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:39.340977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:39.340977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:39.340977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:39.343095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e6466-a61f-4fe4-9a48-e8affc305267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:39.343331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 367/754] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 7 16:04:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:39.344099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e6466-a61f-4fe4-9a48-e8affc305267 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:39.344534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e6466-a61f-4fe4-9a48-e8affc305267 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:39.344978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e6466-a61f-4fe4-9a48-e8affc305267 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:39.348338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b334a-6d94-4ca2-8d42-412e63682f78 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:39.469622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e6466-a61f-4fe4-9a48-e8affc305267 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:39.471627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e6466-a61f-4fe4-9a48-e8affc305267 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:39.498450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:39.581676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 388/755] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 16:04:39.587360 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e6466-a61f-4fe4-9a48-e8affc305267 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:39.625742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b334a-6d94-4ca2-8d42-412e63682f78 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:39.704425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e6466-a61f-4fe4-9a48-e8affc305267 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:39.732216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:39.741939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e755c-79c2-4d57-ad34-482e8d893235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:39.743524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e755c-79c2-4d57-ad34-482e8d893235 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:39.744702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e755c-79c2-4d57-ad34-482e8d893235 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:39.745124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e755c-79c2-4d57-ad34-482e8d893235 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:39.748381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e755c-79c2-4d57-ad34-482e8d893235 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:39.748381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e755c-79c2-4d57-ad34-482e8d893235 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:39.748381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e755c-79c2-4d57-ad34-482e8d893235 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:39.753997 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e55396-0062-4d7e-8606-771f633790e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:39.754878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e55396-0062-4d7e-8606-771f633790e1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:39.755329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e55396-0062-4d7e-8606-771f633790e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:39.755827 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e55396-0062-4d7e-8606-771f633790e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:39.777254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e55396-0062-4d7e-8606-771f633790e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:39.782682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e55396-0062-4d7e-8606-771f633790e1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:39.794258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 368/756] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 7 16:04:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:39.795813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef25af6-374f-433d-b706-eada374cff18 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:39.796211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef25af6-374f-433d-b706-eada374cff18 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:39.796610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef25af6-374f-433d-b706-eada374cff18 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-00e55396-0062-4d7e-8606-771f633790e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:39.801028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:39.801963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:39.803030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:39.803030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:39.803030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:39.803243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 369/757] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:39.804010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef25af6-374f-433d-b706-eada374cff18 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:39.805230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 389/758] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:39.806867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2376e98-e9a6-4a61-af9d-ee47b28f7c55 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:39.807291 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2376e98-e9a6-4a61-af9d-ee47b28f7c55 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:39.807807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2376e98-e9a6-4a61-af9d-ee47b28f7c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:39.835414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-f2376e98-e9a6-4a61-af9d-ee47b28f7c55 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:39.837039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f2376e98-e9a6-4a61-af9d-ee47b28f7c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:39.841656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:39.919253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 390/759] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:39.923811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2376e98-e9a6-4a61-af9d-ee47b28f7c55 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:39.942679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef25af6-374f-433d-b706-eada374cff18 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:40.011703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f0490-bb6f-4634-8704-e60c913b7a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:40.012493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f0490-bb6f-4634-8704-e60c913b7a10 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:40.012776 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f0490-bb6f-4634-8704-e60c913b7a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:40.013050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f0490-bb6f-4634-8704-e60c913b7a10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:40.037117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f0490-bb6f-4634-8704-e60c913b7a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:40.038568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f0490-bb6f-4634-8704-e60c913b7a10 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-e98f0490-bb6f-4634-8704-e60c913b7a10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:40.072587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:40.073753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:40.074678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:40.074678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:40.074678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:40.074678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 370/760] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:40.079589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f0490-bb6f-4634-8704-e60c913b7a10 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:40.133787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2376e98-e9a6-4a61-af9d-ee47b28f7c55 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:40.154271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:40.205525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f328e-e63c-4460-a636-de824070e391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:40.206401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f328e-e63c-4460-a636-de824070e391 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:40.207290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f328e-e63c-4460-a636-de824070e391 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:40.207739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f328e-e63c-4460-a636-de824070e391 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:40.208568 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f328e-e63c-4460-a636-de824070e391 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:40.209392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f328e-e63c-4460-a636-de824070e391 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:40.210408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:40.211493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f328e-e63c-4460-a636-de824070e391 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:40.227614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 391/761] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:40.229127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f328e-e63c-4460-a636-de824070e391 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:40.234713 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f0490-bb6f-4634-8704-e60c913b7a10 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:40.276569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f8896-f8fd-47a4-b412-a2ea9ada44f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:40.277094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f8896-f8fd-47a4-b412-a2ea9ada44f3 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:40.277375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f8896-f8fd-47a4-b412-a2ea9ada44f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:40.277651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f8896-f8fd-47a4-b412-a2ea9ada44f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:40.278291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f8896-f8fd-47a4-b412-a2ea9ada44f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:40.278597 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f8896-f8fd-47a4-b412-a2ea9ada44f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:40.280573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f8896-f8fd-47a4-b412-a2ea9ada44f3 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:40.293182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 371/762] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:40.294074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f8896-f8fd-47a4-b412-a2ea9ada44f3 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:40.355498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f328e-e63c-4460-a636-de824070e391 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:40.391023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c98a4-99a0-4e7a-98e3-97bade719882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:40.392095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c98a4-99a0-4e7a-98e3-97bade719882 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:40.392591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c98a4-99a0-4e7a-98e3-97bade719882 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:40.393059 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c98a4-99a0-4e7a-98e3-97bade719882 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:40.393852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c98a4-99a0-4e7a-98e3-97bade719882 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:40.395420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c98a4-99a0-4e7a-98e3-97bade719882 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:40.398755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f8896-f8fd-47a4-b412-a2ea9ada44f3 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-940c98a4-99a0-4e7a-98e3-97bade719882 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:40.405488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:40.406472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 16:04:40.407294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:40.409981 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 392/763] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Sep 7 16:04:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:40.410956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c98a4-99a0-4e7a-98e3-97bade719882 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:40.442965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ba2f8-7afa-49be-b39c-52e60f720f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:40.443607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ba2f8-7afa-49be-b39c-52e60f720f1d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:40.443920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ba2f8-7afa-49be-b39c-52e60f720f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:40.444198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ba2f8-7afa-49be-b39c-52e60f720f1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:40.452691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ba2f8-7afa-49be-b39c-52e60f720f1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:40.455065 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ba2f8-7afa-49be-b39c-52e60f720f1d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:40.456350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:40.457016 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-0c9ba2f8-7afa-49be-b39c-52e60f720f1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:40.470409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:40.471373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:40.472557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:40.472557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:40.472557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:40.472557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:40.472557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:40.472557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:40.472557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:40.472557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:40.474810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 372/764] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:40.475930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ba2f8-7afa-49be-b39c-52e60f720f1d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:40.533531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c98a4-99a0-4e7a-98e3-97bade719882 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:40.570121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed559a-dcee-4713-8742-d9519bfc0c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:40.571131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed559a-dcee-4713-8742-d9519bfc0c7b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:40.571131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed559a-dcee-4713-8742-d9519bfc0c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:40.571335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed559a-dcee-4713-8742-d9519bfc0c7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:40.578942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed559a-dcee-4713-8742-d9519bfc0c7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:40.580414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed559a-dcee-4713-8742-d9519bfc0c7b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:40.584945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ba2f8-7afa-49be-b39c-52e60f720f1d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-1fed559a-dcee-4713-8742-d9519bfc0c7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:40.592012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:40.592936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:40.594403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:40.594403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:40.594403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:40.594403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:40.594403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:40.594403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:40.594403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:40.594403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:40.594403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 393/765] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:40.594403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed559a-dcee-4713-8742-d9519bfc0c7b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:40.625974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9da05-e29b-44e7-af0a-c01e5b2c28ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:40.627421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9da05-e29b-44e7-af0a-c01e5b2c28ae None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:40.627950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9da05-e29b-44e7-af0a-c01e5b2c28ae None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:40.628453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9da05-e29b-44e7-af0a-c01e5b2c28ae None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:40.629522 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9da05-e29b-44e7-af0a-c01e5b2c28ae None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:40.630116 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9da05-e29b-44e7-af0a-c01e5b2c28ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:40.631885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9da05-e29b-44e7-af0a-c01e5b2c28ae None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:40.643677 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 373/766] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 7 16:04:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:40.644994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9da05-e29b-44e7-af0a-c01e5b2c28ae None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:40.695877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:40.708655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed559a-dcee-4713-8742-d9519bfc0c7b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:40.716480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:40.749641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867f998-b67a-4041-bd68-0a1ecbff0224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:40.750255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867f998-b67a-4041-bd68-0a1ecbff0224 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:40.750813 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867f998-b67a-4041-bd68-0a1ecbff0224 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:40.751169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867f998-b67a-4041-bd68-0a1ecbff0224 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:40.751945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867f998-b67a-4041-bd68-0a1ecbff0224 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:40.752180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867f998-b67a-4041-bd68-0a1ecbff0224 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:40.753624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867f998-b67a-4041-bd68-0a1ecbff0224 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:40.765589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9da05-e29b-44e7-af0a-c01e5b2c28ae None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:40.766700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 394/767] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:40.767588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867f998-b67a-4041-bd68-0a1ecbff0224 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:40.809995 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af51ec9-6f0c-46b7-9df9-8076c83a93d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:40.810526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af51ec9-6f0c-46b7-9df9-8076c83a93d2 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:40.810801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af51ec9-6f0c-46b7-9df9-8076c83a93d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:40.811677 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af51ec9-6f0c-46b7-9df9-8076c83a93d2 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:40.812792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af51ec9-6f0c-46b7-9df9-8076c83a93d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:40.813097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af51ec9-6f0c-46b7-9df9-8076c83a93d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:40.815106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af51ec9-6f0c-46b7-9df9-8076c83a93d2 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:40.829861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 374/768] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:40.830584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af51ec9-6f0c-46b7-9df9-8076c83a93d2 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:40.888503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867f998-b67a-4041-bd68-0a1ecbff0224 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:40.925024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454b113-f504-4943-b14f-57232a001f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:40.925639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454b113-f504-4943-b14f-57232a001f9c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:40.925901 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454b113-f504-4943-b14f-57232a001f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:40.926175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454b113-f504-4943-b14f-57232a001f9c None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:40.926796 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454b113-f504-4943-b14f-57232a001f9c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:40.927091 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454b113-f504-4943-b14f-57232a001f9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:40.928491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454b113-f504-4943-b14f-57232a001f9c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:40.933571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 395/769] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 7 16:04:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:40.934695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531476f-e522-4b5d-85f1-fb579c8083a9 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:40.935029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531476f-e522-4b5d-85f1-fb579c8083a9 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:40.935303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531476f-e522-4b5d-85f1-fb579c8083a9 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:40.937287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 396/770] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:40.937764 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531476f-e522-4b5d-85f1-fb579c8083a9 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:40.953503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:40.962173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af51ec9-6f0c-46b7-9df9-8076c83a93d2 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:40.986553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:41.000890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141224d8-82a1-427b-b1ca-86dfc9f60618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:41.001523 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141224d8-82a1-427b-b1ca-86dfc9f60618 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:41.001833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141224d8-82a1-427b-b1ca-86dfc9f60618 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:41.002078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141224d8-82a1-427b-b1ca-86dfc9f60618 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:41.002876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141224d8-82a1-427b-b1ca-86dfc9f60618 None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=6b6893f49c96479c8f8dd11a372c257c) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:41.003221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141224d8-82a1-427b-b1ca-86dfc9f60618 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:41.007023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141224d8-82a1-427b-b1ca-86dfc9f60618 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:41.014336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 375/771] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:40 2022] GET /identity/v3/role_assignments?user.id=6b6893f49c96479c8f8dd11a372c257c&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:41.014336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141224d8-82a1-427b-b1ca-86dfc9f60618 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:41.079172 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531476f-e522-4b5d-85f1-fb579c8083a9 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:41.126901 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8825a-f638-47a9-a19c-ced66e0610c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:41.127583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8825a-f638-47a9-a19c-ced66e0610c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:41.127908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8825a-f638-47a9-a19c-ced66e0610c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:41.128182 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8825a-f638-47a9-a19c-ced66e0610c4 None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/45061db68b1e40f59935cc9a46d2ff10/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:41.130465 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8825a-f638-47a9-a19c-ced66e0610c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=45061db68b1e40f59935cc9a46d2ff10, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:41.131964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8825a-f638-47a9-a19c-ced66e0610c4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:41.132266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141224d8-82a1-427b-b1ca-86dfc9f60618 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:41.168017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db0be4-3c04-4d7c-8391-dc130096613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:41.168641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db0be4-3c04-4d7c-8391-dc130096613c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:41.168918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db0be4-3c04-4d7c-8391-dc130096613c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:41.169221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db0be4-3c04-4d7c-8391-dc130096613c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:41.170070 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db0be4-3c04-4d7c-8391-dc130096613c None admin] RBAC: Adding query filter params (interface=public, service_id=1b1684c020734d74a78654c7920e7fba, region_id=RegionOne) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:41.170980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db0be4-3c04-4d7c-8391-dc130096613c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:41.172824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db0be4-3c04-4d7c-8391-dc130096613c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:41.191056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 376/772] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 7 16:04:41 2022] GET /identity/v3/endpoints?service_id=1b1684c020734d74a78654c7920e7fba&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:41.191056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb2036-04dc-4ce9-9c64-1d5ce96c58f7 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:41.191056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb2036-04dc-4ce9-9c64-1d5ce96c58f7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:41.191056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb2036-04dc-4ce9-9c64-1d5ce96c58f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:41.212150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-1ddb2036-04dc-4ce9-9c64-1d5ce96c58f7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:41.213379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1ddb2036-04dc-4ce9-9c64-1d5ce96c58f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:41.213867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 397/773] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:04:40 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/45061db68b1e40f59935cc9a46d2ff10/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:41.219800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f3cb4-06a9-40f6-aaa1-af4d9ec79546 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:41.219800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f3cb4-06a9-40f6-aaa1-af4d9ec79546 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:41.219800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f3cb4-06a9-40f6-aaa1-af4d9ec79546 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:41.219800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 398/774] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:41.219800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f3cb4-06a9-40f6-aaa1-af4d9ec79546 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:41.304583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:41.313463 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 377/775] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:41.315104 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce53f91-be21-43c0-af85-f19296af1395 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:41.315387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce53f91-be21-43c0-af85-f19296af1395 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:41.315657 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce53f91-be21-43c0-af85-f19296af1395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:41.322485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:41.357655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f3cb4-06a9-40f6-aaa1-af4d9ec79546 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:41.360927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7ce53f91-be21-43c0-af85-f19296af1395 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:41.362212 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7ce53f91-be21-43c0-af85-f19296af1395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:41.397036 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb4020-1f2b-4760-80f4-4190227df096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:41.402291 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb4020-1f2b-4760-80f4-4190227df096 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:41.402291 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb4020-1f2b-4760-80f4-4190227df096 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:41.402291 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb4020-1f2b-4760-80f4-4190227df096 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:41.402291 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb4020-1f2b-4760-80f4-4190227df096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:41.402291 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb4020-1f2b-4760-80f4-4190227df096 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:41.407329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 399/776] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 7 16:04:41 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:41.472752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 378/777] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:41.637986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb4020-1f2b-4760-80f4-4190227df096 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:41.749546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb4020-1f2b-4760-80f4-4190227df096 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:41.787808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce53f91-be21-43c0-af85-f19296af1395 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:41.804318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce60af-445a-4ace-adb3-bb4528ab7e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:41.805571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce60af-445a-4ace-adb3-bb4528ab7e79 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:41.806056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce60af-445a-4ace-adb3-bb4528ab7e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:41.807784 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce60af-445a-4ace-adb3-bb4528ab7e79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:41.824106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:41.831722 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce60af-445a-4ace-adb3-bb4528ab7e79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-47ce60af-445a-4ace-adb3-bb4528ab7e79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:41.836641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:41.848204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 400/778] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:41.849233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce60af-445a-4ace-adb3-bb4528ab7e79 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:41.969454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce60af-445a-4ace-adb3-bb4528ab7e79 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:41.983266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce53f91-be21-43c0-af85-f19296af1395 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:42.000657 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:42.011796 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bcb6a-a88d-41e6-93e1-11605bfe4b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:42.012599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bcb6a-a88d-41e6-93e1-11605bfe4b9d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:42.013129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bcb6a-a88d-41e6-93e1-11605bfe4b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:42.013543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99bcb6a-a88d-41e6-93e1-11605bfe4b9d None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:42.014924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bcb6a-a88d-41e6-93e1-11605bfe4b9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:42.015406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bcb6a-a88d-41e6-93e1-11605bfe4b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:42.016935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99bcb6a-a88d-41e6-93e1-11605bfe4b9d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:42.026535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 401/779] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:42.032523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bcb6a-a88d-41e6-93e1-11605bfe4b9d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:42.054102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545a978-1023-46f1-a670-d288913280f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:42.056369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545a978-1023-46f1-a670-d288913280f2 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:42.056813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545a978-1023-46f1-a670-d288913280f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:42.061504 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7545a978-1023-46f1-a670-d288913280f2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:42.061504 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545a978-1023-46f1-a670-d288913280f2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:42.061504 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7545a978-1023-46f1-a670-d288913280f2 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-7545a978-1023-46f1-a670-d288913280f2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:42.070347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:42.071335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 16:04:42.072186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:42.074570 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 379/780] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:41 2022] GET /identity/v3/services/network => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:42.080822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545a978-1023-46f1-a670-d288913280f2 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:42.182398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:42.227753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99bcb6a-a88d-41e6-93e1-11605bfe4b9d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:42.243246 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7545a978-1023-46f1-a670-d288913280f2 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:42.285330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6dd296-6d8e-458d-8396-139da064b1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:42.287002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dd296-6d8e-458d-8396-139da064b1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:42.288118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dd296-6d8e-458d-8396-139da064b1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:42.288594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dd296-6d8e-458d-8396-139da064b1e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:42.301828 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdadb3-82fc-4791-ad06-28e8601d775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:42.302736 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdadb3-82fc-4791-ad06-28e8601d775b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:42.304165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdadb3-82fc-4791-ad06-28e8601d775b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:42.304449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bdadb3-82fc-4791-ad06-28e8601d775b None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:42.305109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdadb3-82fc-4791-ad06-28e8601d775b None admin] RBAC: Adding query filter params (name=network) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:42.305674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdadb3-82fc-4791-ad06-28e8601d775b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:42.307836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bdadb3-82fc-4791-ad06-28e8601d775b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:42.308400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6dd296-6d8e-458d-8396-139da064b1e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:42.310138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6dd296-6d8e-458d-8396-139da064b1e2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:42.320609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 380/781] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:04:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:42.325603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdadb3-82fc-4791-ad06-28e8601d775b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-6a6dd296-6d8e-458d-8396-139da064b1e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:42.327346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:42.328277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:42.329168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:42.329168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:42.329168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:42.331214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 402/782] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:42.339410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6dd296-6d8e-458d-8396-139da064b1e2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:42.388101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:42.478323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bdadb3-82fc-4791-ad06-28e8601d775b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:42.520145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6dd296-6d8e-458d-8396-139da064b1e2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:42.541889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:42.559886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c1e4f-f44d-4b09-86ab-8e72248dd9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:42.560822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c1e4f-f44d-4b09-86ab-8e72248dd9bd None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:42.561783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c1e4f-f44d-4b09-86ab-8e72248dd9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:42.562202 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c1e4f-f44d-4b09-86ab-8e72248dd9bd None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:42.563179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c1e4f-f44d-4b09-86ab-8e72248dd9bd None admin] RBAC: Adding query filter params (type=network) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:42.563648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c1e4f-f44d-4b09-86ab-8e72248dd9bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:42.566233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1c1e4f-f44d-4b09-86ab-8e72248dd9bd None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:42.575862 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 381/783] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:04:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:42.584272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554ef96-0e71-4cba-9e15-1b3915ab4bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:42.584892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554ef96-0e71-4cba-9e15-1b3915ab4bdd None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:42.585173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554ef96-0e71-4cba-9e15-1b3915ab4bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:42.585449 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554ef96-0e71-4cba-9e15-1b3915ab4bdd None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:42.586089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554ef96-0e71-4cba-9e15-1b3915ab4bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:42.586394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554ef96-0e71-4cba-9e15-1b3915ab4bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:42.588129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c1e4f-f44d-4b09-86ab-8e72248dd9bd None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:42.588421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554ef96-0e71-4cba-9e15-1b3915ab4bdd None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:42.605883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 403/784] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:42.617842 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554ef96-0e71-4cba-9e15-1b3915ab4bdd None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:42.723236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554ef96-0e71-4cba-9e15-1b3915ab4bdd None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:42.767294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c1e4f-f44d-4b09-86ab-8e72248dd9bd None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:42.774878 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:42.788434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdbd22-2fd6-4d34-ad86-311a8a909cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:42.789820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdbd22-2fd6-4d34-ad86-311a8a909cbf None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:42.789820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdbd22-2fd6-4d34-ad86-311a8a909cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:42.789820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdbd22-2fd6-4d34-ad86-311a8a909cbf None admin] PATH_INFO: `/v3/users/swift` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:42.805585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdbd22-2fd6-4d34-ad86-311a8a909cbf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:42.805585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdbd22-2fd6-4d34-ad86-311a8a909cbf None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-cabdbd22-2fd6-4d34-ad86-311a8a909cbf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:42.820278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:42.821256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:42.822204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:42.822204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:42.822204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:42.822204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:42.822204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:42.822204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:42.822204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:42.822204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:42.822204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:42.822204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:42.822204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 404/785] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:42.825392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490311-9597-4b33-8eae-9adaeb9d6782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:42.826128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490311-9597-4b33-8eae-9adaeb9d6782 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:42.826548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490311-9597-4b33-8eae-9adaeb9d6782 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:42.826918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc490311-9597-4b33-8eae-9adaeb9d6782 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:42.827736 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc490311-9597-4b33-8eae-9adaeb9d6782 None admin] RBAC: Adding query filter params (interface=public, service_id=51a397c5cb4e42b79abd2a74d94af2fa, region_id=RegionOne) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:42.828195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc490311-9597-4b33-8eae-9adaeb9d6782 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:42.829503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdbd22-2fd6-4d34-ad86-311a8a909cbf None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:42.831307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc490311-9597-4b33-8eae-9adaeb9d6782 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:42.848622 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 382/786] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 7 16:04:42 2022] GET /identity/v3/endpoints?service_id=51a397c5cb4e42b79abd2a74d94af2fa&interface=public®ion_id=RegionOne => generated 194 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:42.855702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490311-9597-4b33-8eae-9adaeb9d6782 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:42.871315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:42.957221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdbd22-2fd6-4d34-ad86-311a8a909cbf None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:43.021086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bab169-b551-4015-9b18-d1581fdad33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:43.021631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bab169-b551-4015-9b18-d1581fdad33b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:43.021912 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bab169-b551-4015-9b18-d1581fdad33b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:43.022185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bab169-b551-4015-9b18-d1581fdad33b None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:43.022985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bab169-b551-4015-9b18-d1581fdad33b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:43.023294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bab169-b551-4015-9b18-d1581fdad33b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:43.024779 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bab169-b551-4015-9b18-d1581fdad33b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:43.055555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc490311-9597-4b33-8eae-9adaeb9d6782 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:43.065758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 405/787] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 7 16:04:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:43.071180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bab169-b551-4015-9b18-d1581fdad33b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:43.116356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5443103-8597-4412-b35e-3d2760c70611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:43.116973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5443103-8597-4412-b35e-3d2760c70611 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:43.117252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5443103-8597-4412-b35e-3d2760c70611 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:43.117527 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5443103-8597-4412-b35e-3d2760c70611 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:43.118141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5443103-8597-4412-b35e-3d2760c70611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:43.119504 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5443103-8597-4412-b35e-3d2760c70611 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:43.131958 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 383/788] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 7 16:04:42 2022] GET /identity/v3/services => generated 772 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:43.192825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:43.203913 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bab169-b551-4015-9b18-d1581fdad33b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:43.240585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d975c18-5605-444e-92b9-a0a1ead2ad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:43.241622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d975c18-5605-444e-92b9-a0a1ead2ad8b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:43.242105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d975c18-5605-444e-92b9-a0a1ead2ad8b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:43.242768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d975c18-5605-444e-92b9-a0a1ead2ad8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:43.253875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d975c18-5605-444e-92b9-a0a1ead2ad8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:43.255688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d975c18-5605-444e-92b9-a0a1ead2ad8b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-8d975c18-5605-444e-92b9-a0a1ead2ad8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:43.272841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:43.273823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:43.274704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:43.274704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:43.274704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:43.274704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 406/789] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:43.281702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5443103-8597-4412-b35e-3d2760c70611 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:43.321399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:43.414026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5443103-8597-4412-b35e-3d2760c70611 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:43.466234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63652659-e55a-4994-a26d-a967f99d8800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:43.466873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63652659-e55a-4994-a26d-a967f99d8800 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:43.467607 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82413f7-c278-4d99-965f-936a088233b4 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:43.467701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63652659-e55a-4994-a26d-a967f99d8800 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:43.467903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82413f7-c278-4d99-965f-936a088233b4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:43.468074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63652659-e55a-4994-a26d-a967f99d8800 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:43.468165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82413f7-c278-4d99-965f-936a088233b4 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:43.468669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63652659-e55a-4994-a26d-a967f99d8800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:43.468968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63652659-e55a-4994-a26d-a967f99d8800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:43.470319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63652659-e55a-4994-a26d-a967f99d8800 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:43.470708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 407/790] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:43.475009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691850f-b99a-4efc-80cf-d335a629425c None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:43.475291 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691850f-b99a-4efc-80cf-d335a629425c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:43.475562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691850f-b99a-4efc-80cf-d335a629425c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:43.485074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 384/791] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:43.490618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63652659-e55a-4994-a26d-a967f99d8800 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:43.500098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b691850f-b99a-4efc-80cf-d335a629425c None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:43.501297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b691850f-b99a-4efc-80cf-d335a629425c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:43.584629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 408/792] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:43.626535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:43.641901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63652659-e55a-4994-a26d-a967f99d8800 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:43.682476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90825449-3db6-4d89-8cba-6ec44b4f7922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:43.683123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90825449-3db6-4d89-8cba-6ec44b4f7922 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:43.683399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90825449-3db6-4d89-8cba-6ec44b4f7922 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:43.683671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90825449-3db6-4d89-8cba-6ec44b4f7922 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:43.694205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90825449-3db6-4d89-8cba-6ec44b4f7922 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:43.695642 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90825449-3db6-4d89-8cba-6ec44b4f7922 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-90825449-3db6-4d89-8cba-6ec44b4f7922 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:43.711364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:43.712326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:43.713240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:43.713240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:43.713240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:43.713240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:43.713240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:43.713240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:43.713240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:43.713240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:43.715091 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 385/793] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:43.720074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691850f-b99a-4efc-80cf-d335a629425c None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:43.846692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:43.879101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691850f-b99a-4efc-80cf-d335a629425c None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:43.923749 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46ed21-482f-4396-ab68-f1f7872879c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:43.924729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ed21-482f-4396-ab68-f1f7872879c9 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:43.925230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ed21-482f-4396-ab68-f1f7872879c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:43.925703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ed21-482f-4396-ab68-f1f7872879c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:43.926597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46ed21-482f-4396-ab68-f1f7872879c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:43.927103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46ed21-482f-4396-ab68-f1f7872879c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:43.928741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46ed21-482f-4396-ab68-f1f7872879c9 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:43.941665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b68e8-e276-448b-9806-a18cfccadf4b None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:43.941988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b68e8-e276-448b-9806-a18cfccadf4b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:43.942229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b68e8-e276-448b-9806-a18cfccadf4b None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:43.943566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 409/794] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:43.948106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 386/795] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:43.949491 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b68e8-e276-448b-9806-a18cfccadf4b None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:43.949690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28e0e1-0f69-4984-a8c8-fa5b4e95eed7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:43.949948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28e0e1-0f69-4984-a8c8-fa5b4e95eed7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:43.950234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28e0e1-0f69-4984-a8c8-fa5b4e95eed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:43.970340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-de28e0e1-0f69-4984-a8c8-fa5b4e95eed7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:43.975833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-de28e0e1-0f69-4984-a8c8-fa5b4e95eed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:44.047068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 410/796] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:44.048336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28e0e1-0f69-4984-a8c8-fa5b4e95eed7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:44.100525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b68e8-e276-448b-9806-a18cfccadf4b None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:44.115294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:44.162014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28e0e1-0f69-4984-a8c8-fa5b4e95eed7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:44.197423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c52df-cd25-44fa-a562-e0fea25c2148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:44.198160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c52df-cd25-44fa-a562-e0fea25c2148 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:44.198443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c52df-cd25-44fa-a562-e0fea25c2148 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:44.198774 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c52df-cd25-44fa-a562-e0fea25c2148 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:44.203053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d7e5a-5781-4891-ab2a-d93f0f73374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:44.204015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d7e5a-5781-4891-ab2a-d93f0f73374d None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:44.204494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d7e5a-5781-4891-ab2a-d93f0f73374d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:44.205164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d7e5a-5781-4891-ab2a-d93f0f73374d None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/69a8796195ff4dfb85655cdd91a0229d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:44.207431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d7e5a-5781-4891-ab2a-d93f0f73374d None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=69a8796195ff4dfb85655cdd91a0229d, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:44.209202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d7e5a-5781-4891-ab2a-d93f0f73374d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:44.220626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c52df-cd25-44fa-a562-e0fea25c2148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-2b1c52df-cd25-44fa-a562-e0fea25c2148 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:44.226625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:44.227866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:44.228752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:44.230725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 387/797] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:44.237227 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c52df-cd25-44fa-a562-e0fea25c2148 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:44.251226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:44.282554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 411/798] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:04:44 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/69a8796195ff4dfb85655cdd91a0229d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:44.289689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b197200-6338-42a7-9bc4-d58288d72724 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:44.290345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b197200-6338-42a7-9bc4-d58288d72724 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:44.290809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b197200-6338-42a7-9bc4-d58288d72724 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:44.295156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 412/799] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:44.308644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8ee2c-3ac2-4349-8ba3-41d87be1cff0 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:44.309327 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8ee2c-3ac2-4349-8ba3-41d87be1cff0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:44.309808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f8ee2c-3ac2-4349-8ba3-41d87be1cff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:44.345211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e5f8ee2c-3ac2-4349-8ba3-41d87be1cff0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:44.346532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e5f8ee2c-3ac2-4349-8ba3-41d87be1cff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:44.347677 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c52df-cd25-44fa-a562-e0fea25c2148 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:44.389390 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c43b4-e2a8-4915-8110-f483a711e69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:44.390292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c43b4-e2a8-4915-8110-f483a711e69f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:44.391199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c43b4-e2a8-4915-8110-f483a711e69f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:44.391816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c43b4-e2a8-4915-8110-f483a711e69f None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:44.392809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c43b4-e2a8-4915-8110-f483a711e69f None admin] RBAC: Adding query filter params (name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:44.393299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c43b4-e2a8-4915-8110-f483a711e69f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:44.395649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c43b4-e2a8-4915-8110-f483a711e69f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:44.412453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 388/800] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:44.412995 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c43b4-e2a8-4915-8110-f483a711e69f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:44.450864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 413/801] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:04:44.452289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8ee2c-3ac2-4349-8ba3-41d87be1cff0 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:44.492879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:44.514132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:44.525188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c43b4-e2a8-4915-8110-f483a711e69f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:44.564413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14266a4b-e35a-40c1-a842-1d5dac030d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:44.565047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14266a4b-e35a-40c1-a842-1d5dac030d67 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:44.565513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14266a4b-e35a-40c1-a842-1d5dac030d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:44.565831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14266a4b-e35a-40c1-a842-1d5dac030d67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:44.567245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f8ee2c-3ac2-4349-8ba3-41d87be1cff0 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:44.577993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14266a4b-e35a-40c1-a842-1d5dac030d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-14266a4b-e35a-40c1-a842-1d5dac030d67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:44.583319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:44.584323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:44.585169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 389/802] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:44.590741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14266a4b-e35a-40c1-a842-1d5dac030d67 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:44.610649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ae77-5d55-4985-b80c-626e25a0f13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:44.611613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ae77-5d55-4985-b80c-626e25a0f13d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:44.612168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ae77-5d55-4985-b80c-626e25a0f13d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:44.612777 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ae77-5d55-4985-b80c-626e25a0f13d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:44.624120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4ae77-5d55-4985-b80c-626e25a0f13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:44.625877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4ae77-5d55-4985-b80c-626e25a0f13d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-f6f4ae77-5d55-4985-b80c-626e25a0f13d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:44.643407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:44.645510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:44.646595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:44.646595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:44.646595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:44.649299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 414/803] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:44.655790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ae77-5d55-4985-b80c-626e25a0f13d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:44.702944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14266a4b-e35a-40c1-a842-1d5dac030d67 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:44.734705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:44.740552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48feac62-c8d0-4829-9a21-b9f411b2d410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:44.741277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feac62-c8d0-4829-9a21-b9f411b2d410 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:44.741556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feac62-c8d0-4829-9a21-b9f411b2d410 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:44.741827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feac62-c8d0-4829-9a21-b9f411b2d410 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:44.742493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48feac62-c8d0-4829-9a21-b9f411b2d410 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:44.742795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48feac62-c8d0-4829-9a21-b9f411b2d410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:44.744495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48feac62-c8d0-4829-9a21-b9f411b2d410 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:44.755009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 390/804] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:44.760713 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48feac62-c8d0-4829-9a21-b9f411b2d410 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:44.765090 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ae77-5d55-4985-b80c-626e25a0f13d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:44.768822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:44.805703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd05a3-0b44-4310-8802-5646d85f9c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:44.806578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd05a3-0b44-4310-8802-5646d85f9c26 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:44.806958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd05a3-0b44-4310-8802-5646d85f9c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:44.807330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd05a3-0b44-4310-8802-5646d85f9c26 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:44.808125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd05a3-0b44-4310-8802-5646d85f9c26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:44.808528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd05a3-0b44-4310-8802-5646d85f9c26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:44.810220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd05a3-0b44-4310-8802-5646d85f9c26 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:44.823246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 415/805] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:44.829757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd05a3-0b44-4310-8802-5646d85f9c26 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:44.874221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48feac62-c8d0-4829-9a21-b9f411b2d410 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:44.915725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db57408-d2b5-4acc-b346-0ba75cb37f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:44.916688 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db57408-d2b5-4acc-b346-0ba75cb37f66 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:44.917581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db57408-d2b5-4acc-b346-0ba75cb37f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:44.917863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db57408-d2b5-4acc-b346-0ba75cb37f66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:44.932035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db57408-d2b5-4acc-b346-0ba75cb37f66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:44.934308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db57408-d2b5-4acc-b346-0ba75cb37f66 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:44.953778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd05a3-0b44-4310-8802-5646d85f9c26 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-9db57408-d2b5-4acc-b346-0ba75cb37f66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:44.954789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:44.955697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:44.956623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:44.956623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:44.956623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:44.959152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 391/806] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:44.959152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db57408-d2b5-4acc-b346-0ba75cb37f66 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:45.001598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddeb92-4db0-4c96-9aa8-06eff1c0a03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:45.002298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddeb92-4db0-4c96-9aa8-06eff1c0a03a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:45.002586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddeb92-4db0-4c96-9aa8-06eff1c0a03a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:45.002863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddeb92-4db0-4c96-9aa8-06eff1c0a03a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:45.003462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddeb92-4db0-4c96-9aa8-06eff1c0a03a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:45.006113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddeb92-4db0-4c96-9aa8-06eff1c0a03a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:45.018924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-58ddeb92-4db0-4c96-9aa8-06eff1c0a03a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:45.019622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:45.021013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:45.021887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 416/807] 10.222.0.15 () {56 vars in 1129 bytes} [Wed Sep 7 16:04:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:45.023186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddeb92-4db0-4c96-9aa8-06eff1c0a03a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:45.065226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:45.081571 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db57408-d2b5-4acc-b346-0ba75cb37f66 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:45.117289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c1894-ff25-4bdb-bd82-f260dcd233f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:45.117946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c1894-ff25-4bdb-bd82-f260dcd233f9 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:45.118337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c1894-ff25-4bdb-bd82-f260dcd233f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:45.118678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c1894-ff25-4bdb-bd82-f260dcd233f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:45.127235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c1894-ff25-4bdb-bd82-f260dcd233f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:45.128798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c1894-ff25-4bdb-bd82-f260dcd233f9 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-980c1894-ff25-4bdb-bd82-f260dcd233f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:45.142095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:45.143087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:45.144448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:45.144448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:45.144448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:45.144448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:45.144448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:45.144448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:45.144448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:45.144448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:45.144448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:04:45.144448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:45.148063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 392/808] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 7 16:04:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:45.148974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c1894-ff25-4bdb-bd82-f260dcd233f9 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:45.166875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddeb92-4db0-4c96-9aa8-06eff1c0a03a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:45.215740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af85d7-7dc8-405b-b3a3-8b127c5e8a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:45.217329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af85d7-7dc8-405b-b3a3-8b127c5e8a2e None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:45.217935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af85d7-7dc8-405b-b3a3-8b127c5e8a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:45.218524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af85d7-7dc8-405b-b3a3-8b127c5e8a2e None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:45.219351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af85d7-7dc8-405b-b3a3-8b127c5e8a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:45.219914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af85d7-7dc8-405b-b3a3-8b127c5e8a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:45.222294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af85d7-7dc8-405b-b3a3-8b127c5e8a2e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:45.237201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 417/809] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:45.238841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af85d7-7dc8-405b-b3a3-8b127c5e8a2e None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:45.262739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c1894-ff25-4bdb-bd82-f260dcd233f9 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:45.272206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:45.304527 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3e31b-8366-4050-a089-9b1c40614daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:45.305824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3e31b-8366-4050-a089-9b1c40614daf None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:45.307198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3e31b-8366-4050-a089-9b1c40614daf None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:45.307728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f3e31b-8366-4050-a089-9b1c40614daf None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:45.308599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3e31b-8366-4050-a089-9b1c40614daf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:45.309096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3e31b-8366-4050-a089-9b1c40614daf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:45.311238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f3e31b-8366-4050-a089-9b1c40614daf None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:45.317486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 393/810] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 7 16:04:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:45.318640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3e31b-8366-4050-a089-9b1c40614daf None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:45.435043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f3e31b-8366-4050-a089-9b1c40614daf None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:45.454344 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:45.467178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af85d7-7dc8-405b-b3a3-8b127c5e8a2e None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:45.474550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1a77-fb9a-4224-818f-5cf3f49965ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:45.475468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1a77-fb9a-4224-818f-5cf3f49965ed None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:45.476013 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1a77-fb9a-4224-818f-5cf3f49965ed None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:45.476471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a1a77-fb9a-4224-818f-5cf3f49965ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:45.484761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a1a77-fb9a-4224-818f-5cf3f49965ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:45.486412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304a1a77-fb9a-4224-818f-5cf3f49965ed None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-304a1a77-fb9a-4224-818f-5cf3f49965ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:45.501635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:45.502599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:45.503554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:45.503554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:45.503554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:45.503554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:45.503554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:45.503554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:45.503554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:45.503554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:45.503554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:04:45.503554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:45.510101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 394/811] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 7 16:04:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:45.510101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1a77-fb9a-4224-818f-5cf3f49965ed None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:45.515182 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85f3ca-05f0-4047-bf27-46b68d80d131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:45.516147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f3ca-05f0-4047-bf27-46b68d80d131 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:45.516637 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f3ca-05f0-4047-bf27-46b68d80d131 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:45.517153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85f3ca-05f0-4047-bf27-46b68d80d131 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:45.518140 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85f3ca-05f0-4047-bf27-46b68d80d131 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:45.518651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85f3ca-05f0-4047-bf27-46b68d80d131 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:45.521045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85f3ca-05f0-4047-bf27-46b68d80d131 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:45.543115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:45.550983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 418/812] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 7 16:04:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:45.552139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85f3ca-05f0-4047-bf27-46b68d80d131 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:45.630919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304a1a77-fb9a-4224-818f-5cf3f49965ed None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:45.675452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b3f36-b002-46be-879f-1f4f8a5e0f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:45.676724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b3f36-b002-46be-879f-1f4f8a5e0f80 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:45.677609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b3f36-b002-46be-879f-1f4f8a5e0f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:45.678359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b3f36-b002-46be-879f-1f4f8a5e0f80 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:45.679520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b3f36-b002-46be-879f-1f4f8a5e0f80 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:45.680739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b3f36-b002-46be-879f-1f4f8a5e0f80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:45.682843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b3f36-b002-46be-879f-1f4f8a5e0f80 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:45.686803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85f3ca-05f0-4047-bf27-46b68d80d131 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:45.691220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 395/813] 10.222.0.15 () {56 vars in 1124 bytes} [Wed Sep 7 16:04:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:45.692271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b3f36-b002-46be-879f-1f4f8a5e0f80 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:45.726636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63e500-d549-44f1-9724-1e43d50689bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:45.727295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63e500-d549-44f1-9724-1e43d50689bd None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:45.727590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63e500-d549-44f1-9724-1e43d50689bd None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:45.727885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63e500-d549-44f1-9724-1e43d50689bd None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:45.728644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63e500-d549-44f1-9724-1e43d50689bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:45.728948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63e500-d549-44f1-9724-1e43d50689bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:45.730413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63e500-d549-44f1-9724-1e43d50689bd None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:45.749478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 419/814] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 7 16:04:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:45.750620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63e500-d549-44f1-9724-1e43d50689bd None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:45.755908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:45.807047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:45.814564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b3f36-b002-46be-879f-1f4f8a5e0f80 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:45.851480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20115709-8609-4302-b4ea-c2b939ad6637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:45.852154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115709-8609-4302-b4ea-c2b939ad6637 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:45.852410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63e500-d549-44f1-9724-1e43d50689bd None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:45.852514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115709-8609-4302-b4ea-c2b939ad6637 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:45.852705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115709-8609-4302-b4ea-c2b939ad6637 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:45.864030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20115709-8609-4302-b4ea-c2b939ad6637 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:45.865480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20115709-8609-4302-b4ea-c2b939ad6637 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-20115709-8609-4302-b4ea-c2b939ad6637 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:45.880023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:45.880999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:45.881873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:45.881873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:45.881873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:45.881873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 396/815] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:45.882513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b9e6d-85c7-4c5f-8865-fe06b8b8b2d0 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:45.882608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b9e6d-85c7-4c5f-8865-fe06b8b8b2d0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:45.882876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b9e6d-85c7-4c5f-8865-fe06b8b8b2d0 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:45.884317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 397/816] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:45.889204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b9e6d-85c7-4c5f-8865-fe06b8b8b2d0 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:45.892723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf10b3c-2596-485a-8dd4-aebcd7398861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:45.893334 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf10b3c-2596-485a-8dd4-aebcd7398861 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:45.893614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf10b3c-2596-485a-8dd4-aebcd7398861 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:45.893882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf10b3c-2596-485a-8dd4-aebcd7398861 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:45.894539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf10b3c-2596-485a-8dd4-aebcd7398861 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:45.895914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf10b3c-2596-485a-8dd4-aebcd7398861 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:45.993968 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 420/817] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:04:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:04:45.994720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf10b3c-2596-485a-8dd4-aebcd7398861 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:46.015944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b9e6d-85c7-4c5f-8865-fe06b8b8b2d0 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:46.052472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a24f94-b366-466e-a089-11614cadad33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:46.053397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a24f94-b366-466e-a089-11614cadad33 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:46.053866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a24f94-b366-466e-a089-11614cadad33 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:46.054137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a24f94-b366-466e-a089-11614cadad33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:46.069270 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:46.090476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:46.136200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a24f94-b366-466e-a089-11614cadad33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:46.138107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a24f94-b366-466e-a089-11614cadad33 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:46.143359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf10b3c-2596-485a-8dd4-aebcd7398861 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:46.188406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cad299-59ff-445e-818a-b2adc0389674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:46.189564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cad299-59ff-445e-818a-b2adc0389674 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:46.189938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cad299-59ff-445e-818a-b2adc0389674 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:46.190286 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cad299-59ff-445e-818a-b2adc0389674 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:46.191161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cad299-59ff-445e-818a-b2adc0389674 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:46.191475 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cad299-59ff-445e-818a-b2adc0389674 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:46.192987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cad299-59ff-445e-818a-b2adc0389674 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:46.210036 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 421/818] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:46.211478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b93d63-e705-454d-9463-b042b215d182 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:46.211871 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b93d63-e705-454d-9463-b042b215d182 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:46.212262 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b93d63-e705-454d-9463-b042b215d182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:46.238562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-70b93d63-e705-454d-9463-b042b215d182 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:46.238880 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-70b93d63-e705-454d-9463-b042b215d182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:46.257306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 398/819] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:04:46.258563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a24f94-b366-466e-a089-11614cadad33 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:46.306754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:46.331905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 422/820] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:46.332793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b93d63-e705-454d-9463-b042b215d182 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:46.414589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a24f94-b366-466e-a089-11614cadad33 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:46.433311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:46.479745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055b519-d9c3-4029-b11c-31aa2f9d6910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:46.480442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055b519-d9c3-4029-b11c-31aa2f9d6910 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:46.480723 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055b519-d9c3-4029-b11c-31aa2f9d6910 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:46.480993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055b519-d9c3-4029-b11c-31aa2f9d6910 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:46.491795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055b519-d9c3-4029-b11c-31aa2f9d6910 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:46.494475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055b519-d9c3-4029-b11c-31aa2f9d6910 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-3055b519-d9c3-4029-b11c-31aa2f9d6910 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:46.516325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:46.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:46.518130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:46.518130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:46.518130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:46.518130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:46.518130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:46.518130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:46.518130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:46.518130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:46.518130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 399/821] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:46.525879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055b519-d9c3-4029-b11c-31aa2f9d6910 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:46.528606 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b93d63-e705-454d-9463-b042b215d182 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:46.591610 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac4fc9-5ee5-4453-966c-6457a996fa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:46.596311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac4fc9-5ee5-4453-966c-6457a996fa10 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:46.596585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac4fc9-5ee5-4453-966c-6457a996fa10 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:46.596854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac4fc9-5ee5-4453-966c-6457a996fa10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:46.609067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac4fc9-5ee5-4453-966c-6457a996fa10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:46.610536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac4fc9-5ee5-4453-966c-6457a996fa10 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-92ac4fc9-5ee5-4453-966c-6457a996fa10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:46.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:46.627173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:46.628103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:46.628103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:46.628103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:46.628103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 423/822] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:46.636731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac4fc9-5ee5-4453-966c-6457a996fa10 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:46.697835 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055b519-d9c3-4029-b11c-31aa2f9d6910 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:46.716254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:46.735962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811e9b6-e352-4452-a993-35d8e352ccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:46.736592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811e9b6-e352-4452-a993-35d8e352ccde None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:46.736867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811e9b6-e352-4452-a993-35d8e352ccde None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:46.737140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811e9b6-e352-4452-a993-35d8e352ccde None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:46.737846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811e9b6-e352-4452-a993-35d8e352ccde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:46.739400 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811e9b6-e352-4452-a993-35d8e352ccde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:46.739733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811e9b6-e352-4452-a993-35d8e352ccde None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:46.754002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 400/823] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:46.754860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811e9b6-e352-4452-a993-35d8e352ccde None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:46.760521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac4fc9-5ee5-4453-966c-6457a996fa10 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:46.762156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:46.796898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe45be-2df8-44cc-9f3c-be8d2deff9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:46.798370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe45be-2df8-44cc-9f3c-be8d2deff9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:46.798673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe45be-2df8-44cc-9f3c-be8d2deff9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:46.798954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe45be-2df8-44cc-9f3c-be8d2deff9f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:46.799665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe45be-2df8-44cc-9f3c-be8d2deff9f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:46.799998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe45be-2df8-44cc-9f3c-be8d2deff9f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:46.801379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbe45be-2df8-44cc-9f3c-be8d2deff9f3 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:46.814945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 424/824] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:46.815798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe45be-2df8-44cc-9f3c-be8d2deff9f3 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:46.872477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811e9b6-e352-4452-a993-35d8e352ccde None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:46.909718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c39f1-dcf6-46cb-921c-ddbddd51e351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:46.910031 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c39f1-dcf6-46cb-921c-ddbddd51e351 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:46.910315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c39f1-dcf6-46cb-921c-ddbddd51e351 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:46.910594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c39f1-dcf6-46cb-921c-ddbddd51e351 None admin] PATH_INFO: `/v3/services/network` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:46.911182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c39f1-dcf6-46cb-921c-ddbddd51e351 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:46.912605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c39f1-dcf6-46cb-921c-ddbddd51e351 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-8f8c39f1-dcf6-46cb-921c-ddbddd51e351 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:46.921879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:46.922854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:46.923685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 401/825] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:46 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:46.924518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c39f1-dcf6-46cb-921c-ddbddd51e351 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:46.926761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe45be-2df8-44cc-9f3c-be8d2deff9f3 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:46.956179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:46.971175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa9410-0218-4906-93aa-ce368c7d421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:46.971854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa9410-0218-4906-93aa-ce368c7d421e None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:46.972138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa9410-0218-4906-93aa-ce368c7d421e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:46.972415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa9410-0218-4906-93aa-ce368c7d421e None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/6b6893f49c96479c8f8dd11a372c257c/roles/d9477edba6304c829c6348bc4e6e6c1d` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:46.987954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa9410-0218-4906-93aa-ce368c7d421e None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=6b6893f49c96479c8f8dd11a372c257c, role_id=d9477edba6304c829c6348bc4e6e6c1d)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:46.989410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa9410-0218-4906-93aa-ce368c7d421e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:47.033419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c39f1-dcf6-46cb-921c-ddbddd51e351 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:47.068147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 425/826] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:04:46 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/6b6893f49c96479c8f8dd11a372c257c/roles/d9477edba6304c829c6348bc4e6e6c1d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:47.069633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66930fc5-54a2-4ad9-8826-aa7f60d36532 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:47.069918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66930fc5-54a2-4ad9-8826-aa7f60d36532 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:47.070239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66930fc5-54a2-4ad9-8826-aa7f60d36532 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:47.071801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 426/827] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:47.076094 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66930fc5-54a2-4ad9-8826-aa7f60d36532 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:47.076302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e441b604-d1f2-42be-9a40-9abd7980e293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:47.076302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441b604-d1f2-42be-9a40-9abd7980e293 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:47.076302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441b604-d1f2-42be-9a40-9abd7980e293 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:47.076302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e441b604-d1f2-42be-9a40-9abd7980e293 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:47.084310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441b604-d1f2-42be-9a40-9abd7980e293 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:47.085855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e441b604-d1f2-42be-9a40-9abd7980e293 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-e441b604-d1f2-42be-9a40-9abd7980e293 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:47.098117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:47.099119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:47.100663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:47.100663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:47.100663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:47.100663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:47.100663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:47.100663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:47.100663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:47.100663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:47.100663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 402/828] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:47.101160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265bad9-50b5-4e1f-b88a-068649066e9f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:47.101160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265bad9-50b5-4e1f-b88a-068649066e9f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:47.101472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265bad9-50b5-4e1f-b88a-068649066e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:47.111202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:47.122995 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0265bad9-50b5-4e1f-b88a-068649066e9f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:47.124913 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0265bad9-50b5-4e1f-b88a-068649066e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:47.211954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 403/829] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:47.213678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265bad9-50b5-4e1f-b88a-068649066e9f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:47.234129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:47.241472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66930fc5-54a2-4ad9-8826-aa7f60d36532 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:47.286820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01654b9-4e20-4834-94d8-ad4e5fa8dfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:47.288294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01654b9-4e20-4834-94d8-ad4e5fa8dfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:47.288780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01654b9-4e20-4834-94d8-ad4e5fa8dfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:47.289259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01654b9-4e20-4834-94d8-ad4e5fa8dfb8 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:47.290082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01654b9-4e20-4834-94d8-ad4e5fa8dfb8 None admin] RBAC: Adding query filter params (name=network) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:47.290599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01654b9-4e20-4834-94d8-ad4e5fa8dfb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:47.292181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01654b9-4e20-4834-94d8-ad4e5fa8dfb8 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:47.302134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 427/830] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:04:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:47.303545 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01654b9-4e20-4834-94d8-ad4e5fa8dfb8 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:47.335993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265bad9-50b5-4e1f-b88a-068649066e9f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:47.385224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c4141-10cd-4ef2-b148-705233ddb7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:47.385844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4141-10cd-4ef2-b148-705233ddb7fe None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:47.386120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4141-10cd-4ef2-b148-705233ddb7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:47.386392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c4141-10cd-4ef2-b148-705233ddb7fe None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:47.386945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:47.387089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c4141-10cd-4ef2-b148-705233ddb7fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:47.387396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c4141-10cd-4ef2-b148-705233ddb7fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:47.388892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c4141-10cd-4ef2-b148-705233ddb7fe None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:47.411470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01654b9-4e20-4834-94d8-ad4e5fa8dfb8 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:47.422854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 404/831] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:47.429080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c4141-10cd-4ef2-b148-705233ddb7fe None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:47.449167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd74f6-34b5-4ee6-b4e2-992e5085aaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:47.450021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd74f6-34b5-4ee6-b4e2-992e5085aaac None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:47.450448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd74f6-34b5-4ee6-b4e2-992e5085aaac None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:47.450867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd74f6-34b5-4ee6-b4e2-992e5085aaac None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:47.451693 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd74f6-34b5-4ee6-b4e2-992e5085aaac None admin] RBAC: Adding query filter params (type=network) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:47.452169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd74f6-34b5-4ee6-b4e2-992e5085aaac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:47.453690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd74f6-34b5-4ee6-b4e2-992e5085aaac None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:47.463456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 428/832] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:04:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:47.464819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd74f6-34b5-4ee6-b4e2-992e5085aaac None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:47.567559 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:47.569787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd74f6-34b5-4ee6-b4e2-992e5085aaac None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:47.588971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c4141-10cd-4ef2-b148-705233ddb7fe None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:47.606325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8428b7e-1b14-4685-9f04-a35aadeeaefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:47.607138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8428b7e-1b14-4685-9f04-a35aadeeaefe None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:47.608144 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8428b7e-1b14-4685-9f04-a35aadeeaefe None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:47.608590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8428b7e-1b14-4685-9f04-a35aadeeaefe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:47.609436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8428b7e-1b14-4685-9f04-a35aadeeaefe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:47.610942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8428b7e-1b14-4685-9f04-a35aadeeaefe None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:47.621875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:47.625204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccba27-9e63-4bb8-8e35-e283697c6fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:47.625811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccba27-9e63-4bb8-8e35-e283697c6fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:47.626092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccba27-9e63-4bb8-8e35-e283697c6fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:47.626371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccba27-9e63-4bb8-8e35-e283697c6fb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:47.627092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccba27-9e63-4bb8-8e35-e283697c6fb4 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=45061db68b1e40f59935cc9a46d2ff10) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:47.627402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccba27-9e63-4bb8-8e35-e283697c6fb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:47.628848 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccba27-9e63-4bb8-8e35-e283697c6fb4 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:47.634902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 405/833] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:47 2022] GET /identity/v3/role_assignments?user.id=45061db68b1e40f59935cc9a46d2ff10&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:47.635466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccba27-9e63-4bb8-8e35-e283697c6fb4 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:47.748542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 429/834] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:04:47 2022] POST /identity/v3/endpoints => generated 337 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:04:47.768777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccba27-9e63-4bb8-8e35-e283697c6fb4 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:47.819466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597a849-c455-4fd8-80b4-21f98fef0b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:47.820192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597a849-c455-4fd8-80b4-21f98fef0b90 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:47.820511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597a849-c455-4fd8-80b4-21f98fef0b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:47.820840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597a849-c455-4fd8-80b4-21f98fef0b90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:47.824110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:47.834574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597a849-c455-4fd8-80b4-21f98fef0b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-4597a849-c455-4fd8-80b4-21f98fef0b90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:47.839735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:47.840727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:47.842422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 406/835] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:47.848060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8428b7e-1b14-4685-9f04-a35aadeeaefe None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:47.976637 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8428b7e-1b14-4685-9f04-a35aadeeaefe None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:48.025222 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4d04b-c9ff-4828-9600-b3c8c39cc7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:48.025222 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4d04b-c9ff-4828-9600-b3c8c39cc7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:48.025222 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4d04b-c9ff-4828-9600-b3c8c39cc7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:48.025675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4d04b-c9ff-4828-9600-b3c8c39cc7a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:48.026357 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4d04b-c9ff-4828-9600-b3c8c39cc7a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:48.026726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4d04b-c9ff-4828-9600-b3c8c39cc7a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:48.028581 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4d04b-c9ff-4828-9600-b3c8c39cc7a2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:48.038691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 430/836] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:04:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:48.044366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597a849-c455-4fd8-80b4-21f98fef0b90 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:48.165256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597a849-c455-4fd8-80b4-21f98fef0b90 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:48.207083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f957d-845c-4c1a-a60b-575132560e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:48.207736 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f957d-845c-4c1a-a60b-575132560e92 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:48.208049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f957d-845c-4c1a-a60b-575132560e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:48.208328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f957d-845c-4c1a-a60b-575132560e92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:48.221755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f957d-845c-4c1a-a60b-575132560e92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:48.223286 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f957d-845c-4c1a-a60b-575132560e92 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-228f957d-845c-4c1a-a60b-575132560e92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:48.241222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:48.242248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:48.243732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:48.243732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:48.243732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:48.243732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 407/837] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:48.248895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4d04b-c9ff-4828-9600-b3c8c39cc7a2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:48.321404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:48.365474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4d04b-c9ff-4828-9600-b3c8c39cc7a2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:48.400897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545de4aa-a655-4467-a18f-0ec99aff434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:48.401912 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de4aa-a655-4467-a18f-0ec99aff434b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:48.402426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de4aa-a655-4467-a18f-0ec99aff434b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:48.402899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545de4aa-a655-4467-a18f-0ec99aff434b None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:48.403739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545de4aa-a655-4467-a18f-0ec99aff434b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:48.404359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545de4aa-a655-4467-a18f-0ec99aff434b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:48.405928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545de4aa-a655-4467-a18f-0ec99aff434b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:48.418806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 431/838] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:48.422398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7804475-32b3-4d26-96c1-761e129fff7a None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:48.422398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7804475-32b3-4d26-96c1-761e129fff7a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:48.422398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7804475-32b3-4d26-96c1-761e129fff7a None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:48.422398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 408/839] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:48.425878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545de4aa-a655-4467-a18f-0ec99aff434b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:48.427325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227df28-1b0a-4791-9156-6d387a10fcf0 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:48.427610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227df28-1b0a-4791-9156-6d387a10fcf0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:48.427913 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227df28-1b0a-4791-9156-6d387a10fcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:48.452743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c227df28-1b0a-4791-9156-6d387a10fcf0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:48.454287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c227df28-1b0a-4791-9156-6d387a10fcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:48.511425 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:48.539951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 409/840] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:04:48.559165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545de4aa-a655-4467-a18f-0ec99aff434b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:48.578141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:48.596793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21035516-f022-4db5-9497-7123b51df898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:48.597751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21035516-f022-4db5-9497-7123b51df898 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:48.598310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21035516-f022-4db5-9497-7123b51df898 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:48.598781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21035516-f022-4db5-9497-7123b51df898 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:48.606390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21035516-f022-4db5-9497-7123b51df898 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:48.608700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21035516-f022-4db5-9497-7123b51df898 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-21035516-f022-4db5-9497-7123b51df898 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:48.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:48.622974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:48.624559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:48.624559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:48.624559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:48.624559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:48.624559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:48.624559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:48.624559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:48.624559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:48.624559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 16:04:48.624559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:48.631470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 432/841] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:04:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:48.633123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227df28-1b0a-4791-9156-6d387a10fcf0 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:48.744550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227df28-1b0a-4791-9156-6d387a10fcf0 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:48.783949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b0dae-b62c-4434-976d-077d21ac6cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:48.784561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b0dae-b62c-4434-976d-077d21ac6cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:48.784840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b0dae-b62c-4434-976d-077d21ac6cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:48.785114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5b0dae-b62c-4434-976d-077d21ac6cb5 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:48.785840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b0dae-b62c-4434-976d-077d21ac6cb5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:48.786148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b0dae-b62c-4434-976d-077d21ac6cb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:48.787574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5b0dae-b62c-4434-976d-077d21ac6cb5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:48.806246 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 410/842] 10.222.0.15 () {56 vars in 1135 bytes} [Wed Sep 7 16:04:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:48.812483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21035516-f022-4db5-9497-7123b51df898 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:48.845275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b0dae-b62c-4434-976d-077d21ac6cb5 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:48.883971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:48.939134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21035516-f022-4db5-9497-7123b51df898 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:48.974460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb60fb1-50aa-4fa8-9269-cd54ca50b4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:48.975060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb60fb1-50aa-4fa8-9269-cd54ca50b4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:48.975335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb60fb1-50aa-4fa8-9269-cd54ca50b4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:48.975658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb60fb1-50aa-4fa8-9269-cd54ca50b4d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:48.980998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5b0dae-b62c-4434-976d-077d21ac6cb5 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:49.036504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:49.039025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b34a5-7754-4ced-982e-8aec6c02c285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:49.039791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b34a5-7754-4ced-982e-8aec6c02c285 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:49.040316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b34a5-7754-4ced-982e-8aec6c02c285 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:49.040690 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b34a5-7754-4ced-982e-8aec6c02c285 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:49.041668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b34a5-7754-4ced-982e-8aec6c02c285 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:49.043129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b34a5-7754-4ced-982e-8aec6c02c285 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-602b34a5-7754-4ced-982e-8aec6c02c285 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:49.056393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:49.057837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 16:04:49.058789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:49.059522 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 411/843] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:04:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:49.065241 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b34a5-7754-4ced-982e-8aec6c02c285 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:49.066234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb60fb1-50aa-4fa8-9269-cd54ca50b4d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:49.068048 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb60fb1-50aa-4fa8-9269-cd54ca50b4d2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:49.188818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:49.194340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b34a5-7754-4ced-982e-8aec6c02c285 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:49.196418 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 433/844] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:48 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:04:49.204146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb60fb1-50aa-4fa8-9269-cd54ca50b4d2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:49.233367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38cf7d-df7e-41c1-a0e4-f8cc372a9214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:49.234651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38cf7d-df7e-41c1-a0e4-f8cc372a9214 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:49.234996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38cf7d-df7e-41c1-a0e4-f8cc372a9214 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:49.235337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38cf7d-df7e-41c1-a0e4-f8cc372a9214 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:49.236070 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38cf7d-df7e-41c1-a0e4-f8cc372a9214 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:49.236590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38cf7d-df7e-41c1-a0e4-f8cc372a9214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:49.238826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38cf7d-df7e-41c1-a0e4-f8cc372a9214 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:49.244181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 412/845] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:04:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:49.249275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38cf7d-df7e-41c1-a0e4-f8cc372a9214 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:49.296823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:49.324669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb60fb1-50aa-4fa8-9269-cd54ca50b4d2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:49.362232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38cf7d-df7e-41c1-a0e4-f8cc372a9214 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:49.372603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1aaca-95d8-434c-848f-fc11622fc13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:49.373916 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1aaca-95d8-434c-848f-fc11622fc13f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:49.374407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1aaca-95d8-434c-848f-fc11622fc13f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:49.375157 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1aaca-95d8-434c-848f-fc11622fc13f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:49.388976 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1aaca-95d8-434c-848f-fc11622fc13f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:49.391354 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1aaca-95d8-434c-848f-fc11622fc13f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:49.401353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9c188-2300-4d9f-a8b0-dae74c3d94a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:49.402061 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9c188-2300-4d9f-a8b0-dae74c3d94a8 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:49.402417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9c188-2300-4d9f-a8b0-dae74c3d94a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:49.402771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9c188-2300-4d9f-a8b0-dae74c3d94a8 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:49.403902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9c188-2300-4d9f-a8b0-dae74c3d94a8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:49.404837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9c188-2300-4d9f-a8b0-dae74c3d94a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:49.406562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9c188-2300-4d9f-a8b0-dae74c3d94a8 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-17a1aaca-95d8-434c-848f-fc11622fc13f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:49.411525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:49.412679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:49.413593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:49.413593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:49.413593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:49.420005 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 413/847] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:04:49 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:49.420421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 434/847] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:49.425034 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9c188-2300-4d9f-a8b0-dae74c3d94a8 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:49.464287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:49.545502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9c188-2300-4d9f-a8b0-dae74c3d94a8 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:49.579785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042a51c-7372-4ea5-a387-5a16bd5c08e6 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:49.580282 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042a51c-7372-4ea5-a387-5a16bd5c08e6 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:49.580282 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042a51c-7372-4ea5-a387-5a16bd5c08e6 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:49.581866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 435/848] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:49.588705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bcea9-4b93-4b97-8dd0-b7d15e3d6322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:49.589552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bcea9-4b93-4b97-8dd0-b7d15e3d6322 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:49.589672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca584737-f771-457c-b1a7-626ad52294fe None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:49.589920 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca584737-f771-457c-b1a7-626ad52294fe None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:49.590071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bcea9-4b93-4b97-8dd0-b7d15e3d6322 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:49.590236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca584737-f771-457c-b1a7-626ad52294fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:49.590391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bcea9-4b93-4b97-8dd0-b7d15e3d6322 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:49.591035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2bcea9-4b93-4b97-8dd0-b7d15e3d6322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:49.591340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2bcea9-4b93-4b97-8dd0-b7d15e3d6322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:49.592893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2bcea9-4b93-4b97-8dd0-b7d15e3d6322 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:49.608472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 414/849] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:49.611963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ca584737-f771-457c-b1a7-626ad52294fe None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:49.613342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ca584737-f771-457c-b1a7-626ad52294fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:49.613695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bcea9-4b93-4b97-8dd0-b7d15e3d6322 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:49.688828 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 436/850] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:04:49.715422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:49.728810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bcea9-4b93-4b97-8dd0-b7d15e3d6322 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:49.764893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d65e15-f0aa-446f-a2dc-3106839efced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:49.765868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d65e15-f0aa-446f-a2dc-3106839efced None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:49.766336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d65e15-f0aa-446f-a2dc-3106839efced None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:49.766789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d65e15-f0aa-446f-a2dc-3106839efced None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:49.773917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d65e15-f0aa-446f-a2dc-3106839efced None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:49.775567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d65e15-f0aa-446f-a2dc-3106839efced None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-29d65e15-f0aa-446f-a2dc-3106839efced None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:49.787114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:49.788050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:49.789049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:49.789049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:49.789049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:49.789049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:49.789049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:49.789049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:49.789049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:49.789049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:49.791468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 415/851] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:49.796900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca584737-f771-457c-b1a7-626ad52294fe None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:49.859217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:49.905611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca584737-f771-457c-b1a7-626ad52294fe None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:49.942744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed552d6-d1cd-4a92-975c-5ba49424e773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:49.943630 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed552d6-d1cd-4a92-975c-5ba49424e773 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:49.944094 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed552d6-d1cd-4a92-975c-5ba49424e773 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:49.944524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed552d6-d1cd-4a92-975c-5ba49424e773 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:49.945355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed552d6-d1cd-4a92-975c-5ba49424e773 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:49.945822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed552d6-d1cd-4a92-975c-5ba49424e773 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:49.947398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed552d6-d1cd-4a92-975c-5ba49424e773 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:49.960212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 437/852] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:49.968965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d65e15-f0aa-446f-a2dc-3106839efced None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:49.988553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed552d6-d1cd-4a92-975c-5ba49424e773 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:50.108760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed552d6-d1cd-4a92-975c-5ba49424e773 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:50.121120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d65e15-f0aa-446f-a2dc-3106839efced None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:50.130213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:50.145579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bf0260-16c7-422a-b7e3-df2762c2b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:50.146209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf0260-16c7-422a-b7e3-df2762c2b852 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:50.146485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf0260-16c7-422a-b7e3-df2762c2b852 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:50.147721 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf0260-16c7-422a-b7e3-df2762c2b852 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:50.157202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bf0260-16c7-422a-b7e3-df2762c2b852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:50.159481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73001a23-7486-4b5e-882e-0a9d96cb8dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:50.160328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73001a23-7486-4b5e-882e-0a9d96cb8dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:50.160755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73001a23-7486-4b5e-882e-0a9d96cb8dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:50.161194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73001a23-7486-4b5e-882e-0a9d96cb8dd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-b6bf0260-16c7-422a-b7e3-df2762c2b852 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:50.162045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:50.163110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:50.164151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 438/853] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:04:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:50.164872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7616a5-b697-441a-8f98-094ac337e328 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:50.164947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7616a5-b697-441a-8f98-094ac337e328 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:50.165059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73001a23-7486-4b5e-882e-0a9d96cb8dd9 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=69a8796195ff4dfb85655cdd91a0229d) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:50.165201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7616a5-b697-441a-8f98-094ac337e328 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:50.165627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73001a23-7486-4b5e-882e-0a9d96cb8dd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:50.167199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73001a23-7486-4b5e-882e-0a9d96cb8dd9 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:50.171069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 439/854] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:50.171873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7616a5-b697-441a-8f98-094ac337e328 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:50.175597 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 416/855] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:49 2022] GET /identity/v3/role_assignments?user.id=69a8796195ff4dfb85655cdd91a0229d&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:50.177094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374876a0-2852-4aa3-9f00-0121a6b376af None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:50.177527 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374876a0-2852-4aa3-9f00-0121a6b376af None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:50.177975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374876a0-2852-4aa3-9f00-0121a6b376af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:50.201363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-374876a0-2852-4aa3-9f00-0121a6b376af None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:50.202960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-374876a0-2852-4aa3-9f00-0121a6b376af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:50.212636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:50.290943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 417/856] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:50.317102 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7616a5-b697-441a-8f98-094ac337e328 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:50.359211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a0ee7-7c23-498e-bceb-ec0cb8caaa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:50.360166 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a0ee7-7c23-498e-bceb-ec0cb8caaa6b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:50.360595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a0ee7-7c23-498e-bceb-ec0cb8caaa6b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:50.361015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0a0ee7-7c23-498e-bceb-ec0cb8caaa6b None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:50.361859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a0ee7-7c23-498e-bceb-ec0cb8caaa6b None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:50.362319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a0ee7-7c23-498e-bceb-ec0cb8caaa6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:50.363952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0a0ee7-7c23-498e-bceb-ec0cb8caaa6b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:50.373897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 440/857] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:04:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:50.379497 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374876a0-2852-4aa3-9f00-0121a6b376af None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:50.484520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374876a0-2852-4aa3-9f00-0121a6b376af None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:50.519874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249f6d8-9094-4349-8302-f84b5ff018a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:50.520510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249f6d8-9094-4349-8302-f84b5ff018a5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:50.520792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249f6d8-9094-4349-8302-f84b5ff018a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:50.521068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249f6d8-9094-4349-8302-f84b5ff018a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:50.532012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249f6d8-9094-4349-8302-f84b5ff018a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:50.533461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8249f6d8-9094-4349-8302-f84b5ff018a5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:50.535217 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-8249f6d8-9094-4349-8302-f84b5ff018a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:50.549733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:50.560380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:50.561273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:50.561273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:50.561273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:50.561273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 418/858] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:50.561509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a0ee7-7c23-498e-bceb-ec0cb8caaa6b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:50.561583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249f6d8-9094-4349-8302-f84b5ff018a5 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:50.605329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:50.676549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249f6d8-9094-4349-8302-f84b5ff018a5 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:50.678557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0a0ee7-7c23-498e-bceb-ec0cb8caaa6b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:50.719816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd12080-e9da-4587-afdc-40e9a1b7d8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:50.719816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd12080-e9da-4587-afdc-40e9a1b7d8ff None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:50.720090 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd12080-e9da-4587-afdc-40e9a1b7d8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:50.720373 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd12080-e9da-4587-afdc-40e9a1b7d8ff None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:50.720967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd12080-e9da-4587-afdc-40e9a1b7d8ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:50.722337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd12080-e9da-4587-afdc-40e9a1b7d8ff None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:50.727501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d3ad7-4f88-421e-9b7c-936a9974fc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:50.728321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d3ad7-4f88-421e-9b7c-936a9974fc0f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:50.728747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d3ad7-4f88-421e-9b7c-936a9974fc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:50.729184 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d3ad7-4f88-421e-9b7c-936a9974fc0f None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:50.729971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d3ad7-4f88-421e-9b7c-936a9974fc0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:50.730429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d3ad7-4f88-421e-9b7c-936a9974fc0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-0dd12080-e9da-4587-afdc-40e9a1b7d8ff None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:50.731506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:50.732674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:50.733938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 441/859] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Sep 7 16:04:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:50.735450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d3ad7-4f88-421e-9b7c-936a9974fc0f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:50.743803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd12080-e9da-4587-afdc-40e9a1b7d8ff None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:50.750042 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 419/860] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:50.755436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d3ad7-4f88-421e-9b7c-936a9974fc0f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:50.819310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:50.865933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd12080-e9da-4587-afdc-40e9a1b7d8ff None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:50.874341 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:50.876060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d3ad7-4f88-421e-9b7c-936a9974fc0f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:50.912518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28cc10-c2d4-4edc-8c8c-9e46d1ac809a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:50.913240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28cc10-c2d4-4edc-8c8c-9e46d1ac809a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:50.913523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28cc10-c2d4-4edc-8c8c-9e46d1ac809a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:50.913800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f28cc10-c2d4-4edc-8c8c-9e46d1ac809a None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:50.914615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28cc10-c2d4-4edc-8c8c-9e46d1ac809a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:50.914924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28cc10-c2d4-4edc-8c8c-9e46d1ac809a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:50.916661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f28cc10-c2d4-4edc-8c8c-9e46d1ac809a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:50.917071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd976a3-4834-47c3-b6bc-bebca7ceffdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:50.917956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd976a3-4834-47c3-b6bc-bebca7ceffdb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:50.918474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd976a3-4834-47c3-b6bc-bebca7ceffdb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:50.918906 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd976a3-4834-47c3-b6bc-bebca7ceffdb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:50.926277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 442/861] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 7 16:04:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:50.927654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28cc10-c2d4-4edc-8c8c-9e46d1ac809a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:50.928537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd976a3-4834-47c3-b6bc-bebca7ceffdb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:50.930230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd976a3-4834-47c3-b6bc-bebca7ceffdb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-3fd976a3-4834-47c3-b6bc-bebca7ceffdb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:50.945394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:50.946360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:50.947299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:50.947299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:50.947299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:50.947299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:50.947299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:50.947299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:50.947299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:50.947299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:50.947299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:04:50.947299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:50.947299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 420/862] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 7 16:04:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:50.954542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd976a3-4834-47c3-b6bc-bebca7ceffdb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:51.067187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f28cc10-c2d4-4edc-8c8c-9e46d1ac809a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:51.088900 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd976a3-4834-47c3-b6bc-bebca7ceffdb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:51.103595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8f1a8-895b-49c0-8b83-396c43fd38ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:51.104259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8f1a8-895b-49c0-8b83-396c43fd38ca None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:51.104509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8f1a8-895b-49c0-8b83-396c43fd38ca None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:51.104773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8f1a8-895b-49c0-8b83-396c43fd38ca None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:51.105390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8f1a8-895b-49c0-8b83-396c43fd38ca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:51.105686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8f1a8-895b-49c0-8b83-396c43fd38ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:51.107122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8f1a8-895b-49c0-8b83-396c43fd38ca None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:51.109861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:51.116199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 443/863] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 7 16:04:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:51.134072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57dfdb-5c82-453b-b9e7-8e5102a2b783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:51.135071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57dfdb-5c82-453b-b9e7-8e5102a2b783 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:51.135555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57dfdb-5c82-453b-b9e7-8e5102a2b783 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:51.136080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57dfdb-5c82-453b-b9e7-8e5102a2b783 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:51.137039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57dfdb-5c82-453b-b9e7-8e5102a2b783 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:51.137578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57dfdb-5c82-453b-b9e7-8e5102a2b783 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:51.139888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57dfdb-5c82-453b-b9e7-8e5102a2b783 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:51.161381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 421/864] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 7 16:04:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:51.171789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8f1a8-895b-49c0-8b83-396c43fd38ca None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:51.215161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:51.292385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8f1a8-895b-49c0-8b83-396c43fd38ca None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:51.332203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba150777-bb49-4ff1-a349-45ac9528a20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:51.332822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba150777-bb49-4ff1-a349-45ac9528a20c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:51.333123 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba150777-bb49-4ff1-a349-45ac9528a20c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:51.333438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba150777-bb49-4ff1-a349-45ac9528a20c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:51.405161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba150777-bb49-4ff1-a349-45ac9528a20c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:51.406654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba150777-bb49-4ff1-a349-45ac9528a20c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:51.440142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:51.508434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 444/865] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:51 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:04:51.514547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57dfdb-5c82-453b-b9e7-8e5102a2b783 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:51.622551 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57dfdb-5c82-453b-b9e7-8e5102a2b783 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:51.660909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc031fd-430c-4b35-9252-2b70a31139c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:51.661540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc031fd-430c-4b35-9252-2b70a31139c0 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:51.661820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc031fd-430c-4b35-9252-2b70a31139c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:51.662096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc031fd-430c-4b35-9252-2b70a31139c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:51.672478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc031fd-430c-4b35-9252-2b70a31139c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:51.673923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc031fd-430c-4b35-9252-2b70a31139c0 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-6fc031fd-430c-4b35-9252-2b70a31139c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:51.689744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:51.690698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:51.691592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:51.691592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:51.691592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:51.691592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 422/866] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:51.697903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba150777-bb49-4ff1-a349-45ac9528a20c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:51.738294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21431305-c06a-42fb-8279-8a425037fcfa None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:51.738593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21431305-c06a-42fb-8279-8a425037fcfa None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:51.738845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21431305-c06a-42fb-8279-8a425037fcfa None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:51.740501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 423/867] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:04:51.745606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de464f31-458d-4a63-a2af-7752fd4f92a4 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:51.745890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de464f31-458d-4a63-a2af-7752fd4f92a4 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:51.746161 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de464f31-458d-4a63-a2af-7752fd4f92a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:51.765698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-de464f31-458d-4a63-a2af-7752fd4f92a4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:51.766820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-de464f31-458d-4a63-a2af-7752fd4f92a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:51.787272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:51.817408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba150777-bb49-4ff1-a349-45ac9528a20c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:51.843844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 424/868] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:04:51.855558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e334d4f-13f6-419f-a365-41cc6f325468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:51.856460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e334d4f-13f6-419f-a365-41cc6f325468 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:51.856936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e334d4f-13f6-419f-a365-41cc6f325468 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:51.857458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e334d4f-13f6-419f-a365-41cc6f325468 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:51.858328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e334d4f-13f6-419f-a365-41cc6f325468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:51.858847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e334d4f-13f6-419f-a365-41cc6f325468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:51.860423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e334d4f-13f6-419f-a365-41cc6f325468 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:51.862129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:51.873963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 445/869] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:51.879702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de464f31-458d-4a63-a2af-7752fd4f92a4 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:51.992143 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de464f31-458d-4a63-a2af-7752fd4f92a4 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:52.034611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516a38c-5e0c-4a8b-adf8-1d484482615e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:52.035178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516a38c-5e0c-4a8b-adf8-1d484482615e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:52.035449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516a38c-5e0c-4a8b-adf8-1d484482615e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:52.035718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516a38c-5e0c-4a8b-adf8-1d484482615e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:52.042533 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516a38c-5e0c-4a8b-adf8-1d484482615e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:52.044052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516a38c-5e0c-4a8b-adf8-1d484482615e None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-a516a38c-5e0c-4a8b-adf8-1d484482615e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:52.055800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:52.056716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:52.057630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:52.057630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:52.057630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:52.057630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:52.057630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:52.057630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:52.057630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:52.057630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:52.057630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 425/870] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:52.063384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e334d4f-13f6-419f-a365-41cc6f325468 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:52.136025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516a38c-5e0c-4a8b-adf8-1d484482615e None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:52.153414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:52.182800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e334d4f-13f6-419f-a365-41cc6f325468 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:52.226870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc71c6-b6cc-4e26-9d45-46c651dbbdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:52.227479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc71c6-b6cc-4e26-9d45-46c651dbbdf0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:52.228105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc71c6-b6cc-4e26-9d45-46c651dbbdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:52.228395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc71c6-b6cc-4e26-9d45-46c651dbbdf0 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:52.229105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc71c6-b6cc-4e26-9d45-46c651dbbdf0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:52.229412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc71c6-b6cc-4e26-9d45-46c651dbbdf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:52.230840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc71c6-b6cc-4e26-9d45-46c651dbbdf0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:52.246314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516a38c-5e0c-4a8b-adf8-1d484482615e None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:52.248655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 446/871] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:52.251541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc71c6-b6cc-4e26-9d45-46c651dbbdf0 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:52.287502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900724f3-01d8-4665-8d79-83789df4db81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:52.288222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900724f3-01d8-4665-8d79-83789df4db81 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:52.288505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900724f3-01d8-4665-8d79-83789df4db81 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:52.289145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900724f3-01d8-4665-8d79-83789df4db81 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:52.290076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900724f3-01d8-4665-8d79-83789df4db81 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:52.291435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900724f3-01d8-4665-8d79-83789df4db81 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:52.322704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:52.365647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc71c6-b6cc-4e26-9d45-46c651dbbdf0 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:52.385803 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 426/872] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:04:52 2022] POST /identity/v3/services => generated 251 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:04:52.417792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750fd203-ac82-4f71-88da-6837e0b183b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:52.418787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fd203-ac82-4f71-88da-6837e0b183b0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:52.419275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fd203-ac82-4f71-88da-6837e0b183b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:52.420375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750fd203-ac82-4f71-88da-6837e0b183b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:52.421308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750fd203-ac82-4f71-88da-6837e0b183b0 None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=6b6893f49c96479c8f8dd11a372c257c) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:52.421822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750fd203-ac82-4f71-88da-6837e0b183b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:52.423576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750fd203-ac82-4f71-88da-6837e0b183b0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:52.435929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 447/873] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:52 2022] GET /identity/v3/role_assignments?user.id=6b6893f49c96479c8f8dd11a372c257c&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:52.500294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e4c4b-24d4-4ce1-a175-298337895f9d None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:52.500994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e4c4b-24d4-4ce1-a175-298337895f9d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:52.501458 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e4c4b-24d4-4ce1-a175-298337895f9d None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:52.503519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 427/874] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:52.509160 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b60854-3008-4c39-8b4a-89f4312f93b0 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:52.509738 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b60854-3008-4c39-8b4a-89f4312f93b0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:52.510188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b60854-3008-4c39-8b4a-89f4312f93b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:52.531502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-17b60854-3008-4c39-8b4a-89f4312f93b0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:52.533131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-17b60854-3008-4c39-8b4a-89f4312f93b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:52.614429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 448/875] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:52.919794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e4c4b-24d4-4ce1-a175-298337895f9d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:53.012326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:53.050365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e4c4b-24d4-4ce1-a175-298337895f9d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:53.097232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9ffbe-c984-4eb4-9006-aa7cdabfaee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:53.097879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9ffbe-c984-4eb4-9006-aa7cdabfaee7 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:53.098157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9ffbe-c984-4eb4-9006-aa7cdabfaee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:53.098424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9ffbe-c984-4eb4-9006-aa7cdabfaee7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:53.099068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9ffbe-c984-4eb4-9006-aa7cdabfaee7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:53.100455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9ffbe-c984-4eb4-9006-aa7cdabfaee7 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-07e9ffbe-c984-4eb4-9006-aa7cdabfaee7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:53.114734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:53.121151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:53.123001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 428/876] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 7 16:04:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:53.123907 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b60854-3008-4c39-8b4a-89f4312f93b0 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:53.215964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:53.278007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b60854-3008-4c39-8b4a-89f4312f93b0 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:53.321767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e7cd2-c274-477c-8a93-3857849ab2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:53.322398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e7cd2-c274-477c-8a93-3857849ab2ea None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:53.322676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e7cd2-c274-477c-8a93-3857849ab2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:53.322970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e7cd2-c274-477c-8a93-3857849ab2ea None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:53.323593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e7cd2-c274-477c-8a93-3857849ab2ea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:53.323922 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e7cd2-c274-477c-8a93-3857849ab2ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:53.325551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e7cd2-c274-477c-8a93-3857849ab2ea None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:53.331623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 449/877] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 7 16:04:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:53.337503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9ffbe-c984-4eb4-9006-aa7cdabfaee7 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:53.487049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e9727-2426-4d17-9700-c233ab51774a None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:53.487239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e9727-2426-4d17-9700-c233ab51774a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:53.487543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e9727-2426-4d17-9700-c233ab51774a None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:53.493000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 450/878] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:53.494114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f0bd-e4a0-48b8-bbda-aaaa44cdb973 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:53.494393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f0bd-e4a0-48b8-bbda-aaaa44cdb973 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:53.494672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872f0bd-e4a0-48b8-bbda-aaaa44cdb973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:53.526095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9ffbe-c984-4eb4-9006-aa7cdabfaee7 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:53.530718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6872f0bd-e4a0-48b8-bbda-aaaa44cdb973 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:53.535507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6872f0bd-e4a0-48b8-bbda-aaaa44cdb973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:53.571039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d87ece-9ab6-4f66-b9ed-e66b211a2c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:53.571647 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87ece-9ab6-4f66-b9ed-e66b211a2c38 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:53.571952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87ece-9ab6-4f66-b9ed-e66b211a2c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:53.576264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d87ece-9ab6-4f66-b9ed-e66b211a2c38 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:53.576264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d87ece-9ab6-4f66-b9ed-e66b211a2c38 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:53.576264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d87ece-9ab6-4f66-b9ed-e66b211a2c38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:53.576264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d87ece-9ab6-4f66-b9ed-e66b211a2c38 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:53.586537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 429/879] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 7 16:04:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:53.646174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 451/880] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:53 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:53.913474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d87ece-9ab6-4f66-b9ed-e66b211a2c38 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:53.916380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:54.022321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d87ece-9ab6-4f66-b9ed-e66b211a2c38 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:54.058139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee28e99-774b-45b4-92b3-d43152ad6c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:54.059122 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee28e99-774b-45b4-92b3-d43152ad6c4d None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:54.059602 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee28e99-774b-45b4-92b3-d43152ad6c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:54.060639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee28e99-774b-45b4-92b3-d43152ad6c4d None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:54.061509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee28e99-774b-45b4-92b3-d43152ad6c4d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:54.063055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee28e99-774b-45b4-92b3-d43152ad6c4d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:54.165345 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 430/881] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:04:53 2022] POST /identity/v3/services => generated 269 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:04:54.689348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0c728-2009-43ce-8ac6-9e5f95cf0bed None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:54.689859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0c728-2009-43ce-8ac6-9e5f95cf0bed None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:54.689935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0c728-2009-43ce-8ac6-9e5f95cf0bed None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:54.691470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 452/882] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:54.696974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e0634-e773-499f-b58b-02bdbaf637b9 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:54.697267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e0634-e773-499f-b58b-02bdbaf637b9 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:54.697552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e0634-e773-499f-b58b-02bdbaf637b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:54.717455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-109e0634-e773-499f-b58b-02bdbaf637b9 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:54.719104 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-109e0634-e773-499f-b58b-02bdbaf637b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:54.782259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:54.798314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 431/883] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:04:54.828991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bca38-0a24-4938-a279-2bccdf74322f None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:54.829292 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bca38-0a24-4938-a279-2bccdf74322f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:54.829545 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4bca38-0a24-4938-a279-2bccdf74322f None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:54.831143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 453/884] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:54.836150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf393d2-92e5-4fd1-ba01-7e789c36b5f4 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:54.836437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf393d2-92e5-4fd1-ba01-7e789c36b5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:54.836715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf393d2-92e5-4fd1-ba01-7e789c36b5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:54.855521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-cdf393d2-92e5-4fd1-ba01-7e789c36b5f4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:54.856737 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-cdf393d2-92e5-4fd1-ba01-7e789c36b5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:54.930383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 432/885] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:55.084011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bca38-0a24-4938-a279-2bccdf74322f None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:55.129335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:55.191102 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4bca38-0a24-4938-a279-2bccdf74322f None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:55.199675 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf393d2-92e5-4fd1-ba01-7e789c36b5f4 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:55.228422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834ebef-a9c2-42fc-b43c-38e8e557f94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:55.229052 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834ebef-a9c2-42fc-b43c-38e8e557f94d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:55.229326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834ebef-a9c2-42fc-b43c-38e8e557f94d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:55.229599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834ebef-a9c2-42fc-b43c-38e8e557f94d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:55.230178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834ebef-a9c2-42fc-b43c-38e8e557f94d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:55.231529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834ebef-a9c2-42fc-b43c-38e8e557f94d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-e834ebef-a9c2-42fc-b43c-38e8e557f94d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:55.242709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:55.243723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:55.244631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 454/886] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:04:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:55.251716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834ebef-a9c2-42fc-b43c-38e8e557f94d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:55.318006 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf393d2-92e5-4fd1-ba01-7e789c36b5f4 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:55.335988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:55.352585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794375f9-b8fa-489a-b219-608de331046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:55.353214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794375f9-b8fa-489a-b219-608de331046a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:55.353489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794375f9-b8fa-489a-b219-608de331046a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:55.353761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794375f9-b8fa-489a-b219-608de331046a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:55.365249 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794375f9-b8fa-489a-b219-608de331046a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-794375f9-b8fa-489a-b219-608de331046a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:04:55.370342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:55.372269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:04:55.373685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:55.376107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834ebef-a9c2-42fc-b43c-38e8e557f94d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:55.378163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:55.380808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 433/887] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 7 16:04:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:55.381771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794375f9-b8fa-489a-b219-608de331046a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:55.424854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dd2a-0884-47db-830d-42cadb8d3ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:55.425914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dd2a-0884-47db-830d-42cadb8d3ced None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:55.426455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dd2a-0884-47db-830d-42cadb8d3ced None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:55.426984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95dd2a-0884-47db-830d-42cadb8d3ced None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:55.427893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95dd2a-0884-47db-830d-42cadb8d3ced None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:55.428515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95dd2a-0884-47db-830d-42cadb8d3ced None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:55.431039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95dd2a-0884-47db-830d-42cadb8d3ced None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:55.442796 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 455/888] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:04:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:55.444072 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dd2a-0884-47db-830d-42cadb8d3ced None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:55.498981 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794375f9-b8fa-489a-b219-608de331046a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:55.544292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e8334-7960-4a21-91e1-8d895f5a9f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:55.545791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e8334-7960-4a21-91e1-8d895f5a9f46 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:55.546286 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e8334-7960-4a21-91e1-8d895f5a9f46 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:55.546804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e8334-7960-4a21-91e1-8d895f5a9f46 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:55.547696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e8334-7960-4a21-91e1-8d895f5a9f46 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:55.548236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e8334-7960-4a21-91e1-8d895f5a9f46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:55.549823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e8334-7960-4a21-91e1-8d895f5a9f46 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:55.558569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 434/889] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 7 16:04:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:55.569013 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e8334-7960-4a21-91e1-8d895f5a9f46 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:55.571805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95dd2a-0884-47db-830d-42cadb8d3ced None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:55.616990 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486872e-2027-4259-b1e5-bd4894402609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:55.617650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486872e-2027-4259-b1e5-bd4894402609 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:55.618077 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486872e-2027-4259-b1e5-bd4894402609 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:55.618491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486872e-2027-4259-b1e5-bd4894402609 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:55.619120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486872e-2027-4259-b1e5-bd4894402609 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:55.620189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486872e-2027-4259-b1e5-bd4894402609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:55.621587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486872e-2027-4259-b1e5-bd4894402609 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:55.627807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 456/890] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:04:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:55.632387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:55.633500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486872e-2027-4259-b1e5-bd4894402609 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:55.673220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:55.696282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e8334-7960-4a21-91e1-8d895f5a9f46 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:55.733766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b81328-29e0-46cd-a587-007804003e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:55.734421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b81328-29e0-46cd-a587-007804003e01 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:55.734696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b81328-29e0-46cd-a587-007804003e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:55.734965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b81328-29e0-46cd-a587-007804003e01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:55.755262 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486872e-2027-4259-b1e5-bd4894402609 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:55.755886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b81328-29e0-46cd-a587-007804003e01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:55.756814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b81328-29e0-46cd-a587-007804003e01 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-26b81328-29e0-46cd-a587-007804003e01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:55.779496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:55.781441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:55.782561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:55.782561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:55.782561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:55.785883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 435/891] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:55.786871 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b81328-29e0-46cd-a587-007804003e01 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:55.803295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1004a0-fb2c-4b0e-8358-1f4d74b00876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:55.804309 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1004a0-fb2c-4b0e-8358-1f4d74b00876 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:55.804798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1004a0-fb2c-4b0e-8358-1f4d74b00876 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:55.805574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1004a0-fb2c-4b0e-8358-1f4d74b00876 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:55.806484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1004a0-fb2c-4b0e-8358-1f4d74b00876 None admin] RBAC: Adding query filter params (interface=public, service_id=a971f598933e4e2d8ece943771b3190b, region_id=RegionOne) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:55.807007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1004a0-fb2c-4b0e-8358-1f4d74b00876 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:55.808684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1004a0-fb2c-4b0e-8358-1f4d74b00876 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:55.821573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 457/892] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 7 16:04:55 2022] GET /identity/v3/endpoints?service_id=a971f598933e4e2d8ece943771b3190b&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:55.822748 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1004a0-fb2c-4b0e-8358-1f4d74b00876 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:55.884685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:55.894515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b81328-29e0-46cd-a587-007804003e01 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:55.922863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:55.929459 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1004a0-fb2c-4b0e-8358-1f4d74b00876 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:55.930603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d98dcdd-9e5d-4ab0-824f-3f1064f2ab79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:55.931279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98dcdd-9e5d-4ab0-824f-3f1064f2ab79 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:55.931564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98dcdd-9e5d-4ab0-824f-3f1064f2ab79 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:55.931828 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d98dcdd-9e5d-4ab0-824f-3f1064f2ab79 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:55.936859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98dcdd-9e5d-4ab0-824f-3f1064f2ab79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:55.936859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98dcdd-9e5d-4ab0-824f-3f1064f2ab79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:55.936859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d98dcdd-9e5d-4ab0-824f-3f1064f2ab79 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:55.951626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 436/893] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:55.951626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056ed0a-f02d-4c85-9c4e-977d39ec5357 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:55.951626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056ed0a-f02d-4c85-9c4e-977d39ec5357 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:55.951626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056ed0a-f02d-4c85-9c4e-977d39ec5357 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:55.951626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 437/894] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:55.952596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056ed0a-f02d-4c85-9c4e-977d39ec5357 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:55.972815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201f9b6-db3f-4009-9193-81db2931623c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:55.973432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201f9b6-db3f-4009-9193-81db2931623c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:55.973710 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201f9b6-db3f-4009-9193-81db2931623c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:55.973983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201f9b6-db3f-4009-9193-81db2931623c None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:55.974587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201f9b6-db3f-4009-9193-81db2931623c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:55.975977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201f9b6-db3f-4009-9193-81db2931623c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:55.981793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 458/895] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 7 16:04:55 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:04:55.982991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e256a-b508-4ccc-b148-44315e2aba6f None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:55.983278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e256a-b508-4ccc-b148-44315e2aba6f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:55.983551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e256a-b508-4ccc-b148-44315e2aba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:56.006439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-215e256a-b508-4ccc-b148-44315e2aba6f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:56.006439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-215e256a-b508-4ccc-b148-44315e2aba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:56.100500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056ed0a-f02d-4c85-9c4e-977d39ec5357 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:56.108102 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 459/896] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:56.144191 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b24889-fcc2-4252-a5b9-17a9e8c6e4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:56.145142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b24889-fcc2-4252-a5b9-17a9e8c6e4cb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:56.145584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b24889-fcc2-4252-a5b9-17a9e8c6e4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:56.146027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b24889-fcc2-4252-a5b9-17a9e8c6e4cb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:56.155794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b24889-fcc2-4252-a5b9-17a9e8c6e4cb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:56.157539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b24889-fcc2-4252-a5b9-17a9e8c6e4cb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-07b24889-fcc2-4252-a5b9-17a9e8c6e4cb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:04:56.170442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:56.171697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:56.172933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:56.172933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:56.172933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:04:56.172933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:04:56.172933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:04:56.172933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:04:56.172933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:04:56.172933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:04:56.172933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:04:56.172933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:56.175246 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 438/897] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 7 16:04:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:56.184936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e256a-b508-4ccc-b148-44315e2aba6f None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:56.280186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:56.292030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e256a-b508-4ccc-b148-44315e2aba6f None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:56.328870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fed88-7ae3-46f6-a31d-882c5b0925cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:56.329546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fed88-7ae3-46f6-a31d-882c5b0925cb None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:56.329803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fed88-7ae3-46f6-a31d-882c5b0925cb None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:56.330083 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fed88-7ae3-46f6-a31d-882c5b0925cb None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:56.330847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fed88-7ae3-46f6-a31d-882c5b0925cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:56.331158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fed88-7ae3-46f6-a31d-882c5b0925cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:56.332689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fed88-7ae3-46f6-a31d-882c5b0925cb None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:56.354515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 460/898] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 7 16:04:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:56.360252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b24889-fcc2-4252-a5b9-17a9e8c6e4cb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:56.377269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:56.391697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fed88-7ae3-46f6-a31d-882c5b0925cb None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:56.499010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b24889-fcc2-4252-a5b9-17a9e8c6e4cb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:56.525935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fed88-7ae3-46f6-a31d-882c5b0925cb None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:56.537856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1081e0-62f5-4aa5-afd9-630e281ad5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:56.538462 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1081e0-62f5-4aa5-afd9-630e281ad5fc None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:56.538741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1081e0-62f5-4aa5-afd9-630e281ad5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:56.539013 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1081e0-62f5-4aa5-afd9-630e281ad5fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:56.563009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd62ae9-62df-4ca6-8f3a-53210e766ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:56.564015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd62ae9-62df-4ca6-8f3a-53210e766ba1 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:56.564434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd62ae9-62df-4ca6-8f3a-53210e766ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:56.564867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd62ae9-62df-4ca6-8f3a-53210e766ba1 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:56.565664 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd62ae9-62df-4ca6-8f3a-53210e766ba1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:56.567346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd62ae9-62df-4ca6-8f3a-53210e766ba1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:56.607121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1081e0-62f5-4aa5-afd9-630e281ad5fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:56.608505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1081e0-62f5-4aa5-afd9-630e281ad5fc None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:56.621788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:56.636259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 461/899] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:04:56 2022] POST /identity/v3/services => generated 241 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:04:56.637962 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0294a-a6ea-45a2-94d5-b1b1f1bc5a3a None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:56.639086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0294a-a6ea-45a2-94d5-b1b1f1bc5a3a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:56.639590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0294a-a6ea-45a2-94d5-b1b1f1bc5a3a None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:56.641477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 462/900] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:56.646667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e420079-aea0-43de-980e-5c4c807a7e8b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:56.648112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e420079-aea0-43de-980e-5c4c807a7e8b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:56.648676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e420079-aea0-43de-980e-5c4c807a7e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:56.666561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:56.675133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8e420079-aea0-43de-980e-5c4c807a7e8b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:56.680556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8e420079-aea0-43de-980e-5c4c807a7e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:56.741961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 439/901] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:04:56 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:04:56.747027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1081e0-62f5-4aa5-afd9-630e281ad5fc None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:56.782961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 463/902] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:56.879475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1081e0-62f5-4aa5-afd9-630e281ad5fc None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:56.893875 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:56.915292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b55ca3-aca0-4e54-9260-e54db6178dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:56.915963 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b55ca3-aca0-4e54-9260-e54db6178dff None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:56.916245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b55ca3-aca0-4e54-9260-e54db6178dff None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:56.916519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b55ca3-aca0-4e54-9260-e54db6178dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:56.928078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b55ca3-aca0-4e54-9260-e54db6178dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:56.929532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b55ca3-aca0-4e54-9260-e54db6178dff None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-e5b55ca3-aca0-4e54-9260-e54db6178dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:04:56.945638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:56.946604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:04:56.948138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:04:56.948138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:04:56.948138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:56.948138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 440/903] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:56.952362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e420079-aea0-43de-980e-5c4c807a7e8b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:57.059314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b55ca3-aca0-4e54-9260-e54db6178dff None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:57.079030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e420079-aea0-43de-980e-5c4c807a7e8b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:57.111082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:57.116836 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0f87e-3297-4f7e-8389-c449ed5323e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:57.117464 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0f87e-3297-4f7e-8389-c449ed5323e2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:57.117745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0f87e-3297-4f7e-8389-c449ed5323e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:57.118021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0f87e-3297-4f7e-8389-c449ed5323e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:57.118672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0f87e-3297-4f7e-8389-c449ed5323e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:57.119068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0f87e-3297-4f7e-8389-c449ed5323e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:57.120508 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0f87e-3297-4f7e-8389-c449ed5323e2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:57.133992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 464/904] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:57.139445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0f87e-3297-4f7e-8389-c449ed5323e2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:57.166326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b55ca3-aca0-4e54-9260-e54db6178dff None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:57.201761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56964d5f-9929-4a74-8ef8-7f9ccd2a3160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:57.202417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56964d5f-9929-4a74-8ef8-7f9ccd2a3160 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:57.202698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56964d5f-9929-4a74-8ef8-7f9ccd2a3160 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:57.202975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56964d5f-9929-4a74-8ef8-7f9ccd2a3160 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:57.203557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56964d5f-9929-4a74-8ef8-7f9ccd2a3160 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:57.204951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56964d5f-9929-4a74-8ef8-7f9ccd2a3160 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-56964d5f-9929-4a74-8ef8-7f9ccd2a3160 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:57.214039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:57.215045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:57.216521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 441/905] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Sep 7 16:04:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:57.224832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56964d5f-9929-4a74-8ef8-7f9ccd2a3160 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:57.267851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:57.274660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0f87e-3297-4f7e-8389-c449ed5323e2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:57.312086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924efc9-6248-4515-90b5-45c10993fe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:57.312716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924efc9-6248-4515-90b5-45c10993fe72 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:57.312994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924efc9-6248-4515-90b5-45c10993fe72 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:57.313265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924efc9-6248-4515-90b5-45c10993fe72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:57.320292 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924efc9-6248-4515-90b5-45c10993fe72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:57.321729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924efc9-6248-4515-90b5-45c10993fe72 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-d924efc9-6248-4515-90b5-45c10993fe72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:04:57.333957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:57.334889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:57.336215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:04:57.336215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:04:57.336215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:04:57.336215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:04:57.336215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:04:57.336215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:04:57.336215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:04:57.336215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:57.336215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 465/906] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:04:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:57.340807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924efc9-6248-4515-90b5-45c10993fe72 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:57.368809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56964d5f-9929-4a74-8ef8-7f9ccd2a3160 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:57.400663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:57.417793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff038b-b1d5-4236-b6de-2477189bf6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:57.418407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff038b-b1d5-4236-b6de-2477189bf6ca None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:57.418680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff038b-b1d5-4236-b6de-2477189bf6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:57.418946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff038b-b1d5-4236-b6de-2477189bf6ca None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:57.419857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff038b-b1d5-4236-b6de-2477189bf6ca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:57.420159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff038b-b1d5-4236-b6de-2477189bf6ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:57.421578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff038b-b1d5-4236-b6de-2477189bf6ca None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:57.432273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 442/907] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 7 16:04:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:57.433188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff038b-b1d5-4236-b6de-2477189bf6ca None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:57.461326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924efc9-6248-4515-90b5-45c10993fe72 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:57.505145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cfc8f-3263-4ea4-9db1-7b795f598098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:57.506323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cfc8f-3263-4ea4-9db1-7b795f598098 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:57.506798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cfc8f-3263-4ea4-9db1-7b795f598098 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:57.507375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cfc8f-3263-4ea4-9db1-7b795f598098 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:57.508471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1cfc8f-3263-4ea4-9db1-7b795f598098 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:57.509246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1cfc8f-3263-4ea4-9db1-7b795f598098 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:57.511394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1cfc8f-3263-4ea4-9db1-7b795f598098 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:57.533592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 466/908] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:04:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:04:57.536135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cfc8f-3263-4ea4-9db1-7b795f598098 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:57.549118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:57.560669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff038b-b1d5-4236-b6de-2477189bf6ca None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:57.605287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e4851-4245-4d82-b402-d235e0eb3edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:57.606284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4851-4245-4d82-b402-d235e0eb3edb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:57.606740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4851-4245-4d82-b402-d235e0eb3edb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:57.607159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e4851-4245-4d82-b402-d235e0eb3edb None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:57.608538 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e4851-4245-4d82-b402-d235e0eb3edb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:57.608846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e4851-4245-4d82-b402-d235e0eb3edb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:57.611202 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e4851-4245-4d82-b402-d235e0eb3edb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:57.621552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 443/909] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 7 16:04:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:57.622514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e4851-4245-4d82-b402-d235e0eb3edb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:57.656400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cfc8f-3263-4ea4-9db1-7b795f598098 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:57.674067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:57.704225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e12f06-be20-4b88-88de-6fdcb9d76849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:57.705212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e12f06-be20-4b88-88de-6fdcb9d76849 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:57.705702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e12f06-be20-4b88-88de-6fdcb9d76849 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:57.706174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e12f06-be20-4b88-88de-6fdcb9d76849 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:57.707632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e12f06-be20-4b88-88de-6fdcb9d76849 None admin] RBAC: Adding query filter params (role.id=2c8c5f15b3624270a26334ed3ebdf44a, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=6b6893f49c96479c8f8dd11a372c257c) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:57.708184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e12f06-be20-4b88-88de-6fdcb9d76849 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:57.710686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e12f06-be20-4b88-88de-6fdcb9d76849 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:57.724378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 467/910] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:04:57 2022] GET /identity/v3/role_assignments?user.id=6b6893f49c96479c8f8dd11a372c257c&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=2c8c5f15b3624270a26334ed3ebdf44a => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:57.740510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e4851-4245-4d82-b402-d235e0eb3edb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:57.779797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59c3b8-fdb0-4e86-989a-5720cec04fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:57.780426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59c3b8-fdb0-4e86-989a-5720cec04fcc None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:57.780701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59c3b8-fdb0-4e86-989a-5720cec04fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:57.780972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59c3b8-fdb0-4e86-989a-5720cec04fcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:57.781675 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59c3b8-fdb0-4e86-989a-5720cec04fcc None admin] RBAC: Adding query filter params (interface=public, service_id=adf5d27b7e6d4b65b230434085960723, region_id=RegionOne) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:57.781980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59c3b8-fdb0-4e86-989a-5720cec04fcc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:57.783411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f59c3b8-fdb0-4e86-989a-5720cec04fcc None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:57.790019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 444/911] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 7 16:04:57 2022] GET /identity/v3/endpoints?service_id=adf5d27b7e6d4b65b230434085960723&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:57.796129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e12f06-be20-4b88-88de-6fdcb9d76849 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:57.914749 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e12f06-be20-4b88-88de-6fdcb9d76849 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:57.952392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c0ccb3-ec86-4d10-97ff-9673d87ccdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:57.953361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0ccb3-ec86-4d10-97ff-9673d87ccdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:57.953876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0ccb3-ec86-4d10-97ff-9673d87ccdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:57.954348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c0ccb3-ec86-4d10-97ff-9673d87ccdf1 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:57.955165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0ccb3-ec86-4d10-97ff-9673d87ccdf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:57.956783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c0ccb3-ec86-4d10-97ff-9673d87ccdf1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:57.963689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 468/912] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 7 16:04:57 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:04:58.328975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22bfd4-73a4-4987-8525-3116ca7eeb70 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:58.329179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22bfd4-73a4-4987-8525-3116ca7eeb70 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:58.329447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22bfd4-73a4-4987-8525-3116ca7eeb70 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:58.331054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 445/913] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:04:58.336278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2e373-77d7-4a64-8d35-cc9f66b0eb2f None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:58.336570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2e373-77d7-4a64-8d35-cc9f66b0eb2f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:58.336851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2e373-77d7-4a64-8d35-cc9f66b0eb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:58.355385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-82f2e373-77d7-4a64-8d35-cc9f66b0eb2f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:58.357165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-82f2e373-77d7-4a64-8d35-cc9f66b0eb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:58.393461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:58.432055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 469/914] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:04:58.711396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22bfd4-73a4-4987-8525-3116ca7eeb70 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:58.736068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:58.826034 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22bfd4-73a4-4987-8525-3116ca7eeb70 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:58.864125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b29eec-3170-4fbf-872c-3d7771666552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:58.865070 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b29eec-3170-4fbf-872c-3d7771666552 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:58.865509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b29eec-3170-4fbf-872c-3d7771666552 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:58.865944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b29eec-3170-4fbf-872c-3d7771666552 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:58.866700 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b29eec-3170-4fbf-872c-3d7771666552 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:58.868310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b29eec-3170-4fbf-872c-3d7771666552 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-f5b29eec-3170-4fbf-872c-3d7771666552 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:58.878428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:58.879672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 16:04:58.880706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:04:58.883155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 446/915] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:04:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:04:58.889426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2e373-77d7-4a64-8d35-cc9f66b0eb2f None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:58.924857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283d6d7-c648-4322-a32d-7387a266e665 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:58.925132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283d6d7-c648-4322-a32d-7387a266e665 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:58.925396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283d6d7-c648-4322-a32d-7387a266e665 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:58.926970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 447/916] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:04:58.931905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3530653-040f-4956-af60-603bb91252c6 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:58.932305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3530653-040f-4956-af60-603bb91252c6 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:58.932646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3530653-040f-4956-af60-603bb91252c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:58.955295 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-e3530653-040f-4956-af60-603bb91252c6 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:58.957153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e3530653-040f-4956-af60-603bb91252c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:59.013701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2e373-77d7-4a64-8d35-cc9f66b0eb2f None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:59.053873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe320f68-7bdd-4441-a1f2-88f101548801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:59.054511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe320f68-7bdd-4441-a1f2-88f101548801 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:59.055045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe320f68-7bdd-4441-a1f2-88f101548801 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:59.055354 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe320f68-7bdd-4441-a1f2-88f101548801 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:59.056070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe320f68-7bdd-4441-a1f2-88f101548801 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:59.056369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe320f68-7bdd-4441-a1f2-88f101548801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:59.057729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe320f68-7bdd-4441-a1f2-88f101548801 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:59.062939 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 470/917] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:04:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:59.068136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe320f68-7bdd-4441-a1f2-88f101548801 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:59.069838 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 448/918] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:04:59.105603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:59.173477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe320f68-7bdd-4441-a1f2-88f101548801 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:59.208794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a69c35-fb54-49a3-b3a0-7570cebfd1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:59.209395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a69c35-fb54-49a3-b3a0-7570cebfd1ef None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:59.209670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a69c35-fb54-49a3-b3a0-7570cebfd1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:59.209940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a69c35-fb54-49a3-b3a0-7570cebfd1ef None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:59.210559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a69c35-fb54-49a3-b3a0-7570cebfd1ef None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:59.210856 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a69c35-fb54-49a3-b3a0-7570cebfd1ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:59.212235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a69c35-fb54-49a3-b3a0-7570cebfd1ef None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:59.218046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 471/919] 10.222.0.15 () {56 vars in 1114 bytes} [Wed Sep 7 16:04:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:59.223395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3530653-040f-4956-af60-603bb91252c6 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:59.239643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:59.330581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3530653-040f-4956-af60-603bb91252c6 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:59.345658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a69c35-fb54-49a3-b3a0-7570cebfd1ef None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:59.367046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf7c6a-76aa-42b2-b7c5-e61f7df2c3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:59.368062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf7c6a-76aa-42b2-b7c5-e61f7df2c3d7 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:59.368527 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf7c6a-76aa-42b2-b7c5-e61f7df2c3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:59.369004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf7c6a-76aa-42b2-b7c5-e61f7df2c3d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:59.369869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf7c6a-76aa-42b2-b7c5-e61f7df2c3d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:59.371546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf7c6a-76aa-42b2-b7c5-e61f7df2c3d7 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:59.449970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:59.454558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a69c35-fb54-49a3-b3a0-7570cebfd1ef None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:59.459988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 449/920] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:04:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:04:59.504830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e4057-5181-4daa-9a96-589a15db6077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:59.505573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e4057-5181-4daa-9a96-589a15db6077 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:59.505850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e4057-5181-4daa-9a96-589a15db6077 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:59.506227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e4057-5181-4daa-9a96-589a15db6077 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:59.506816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e4057-5181-4daa-9a96-589a15db6077 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:59.508371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e4057-5181-4daa-9a96-589a15db6077 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-844e4057-5181-4daa-9a96-589a15db6077 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:04:59.518815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:04:59.519940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:04:59.520795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 472/921] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 7 16:04:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:04:59.527137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf7c6a-76aa-42b2-b7c5-e61f7df2c3d7 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:59.606534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:59.643303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf7c6a-76aa-42b2-b7c5-e61f7df2c3d7 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:59.680754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1345d0-b41b-40dd-9d70-b1bca55f67b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:59.681367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1345d0-b41b-40dd-9d70-b1bca55f67b1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:59.681646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1345d0-b41b-40dd-9d70-b1bca55f67b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:59.681975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1345d0-b41b-40dd-9d70-b1bca55f67b1 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:59.682608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1345d0-b41b-40dd-9d70-b1bca55f67b1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:59.682908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1345d0-b41b-40dd-9d70-b1bca55f67b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:59.684291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1345d0-b41b-40dd-9d70-b1bca55f67b1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:59.690240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 450/922] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 7 16:04:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:59.695822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e4057-5181-4daa-9a96-589a15db6077 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:59.798547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e4057-5181-4daa-9a96-589a15db6077 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:59.833670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61813ea9-8053-4857-8869-3f0d24b95a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:59.834281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61813ea9-8053-4857-8869-3f0d24b95a91 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:59.834555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61813ea9-8053-4857-8869-3f0d24b95a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:59.834825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61813ea9-8053-4857-8869-3f0d24b95a91 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:59.835558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61813ea9-8053-4857-8869-3f0d24b95a91 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:59.835890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61813ea9-8053-4857-8869-3f0d24b95a91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:59.837301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61813ea9-8053-4857-8869-3f0d24b95a91 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:59.842534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 473/923] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 7 16:04:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:59.848363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1345d0-b41b-40dd-9d70-b1bca55f67b1 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:59.959564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1345d0-b41b-40dd-9d70-b1bca55f67b1 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:59.985786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1ea5b-e889-4b00-baa0-8405d367b722 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:59.986293 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1ea5b-e889-4b00-baa0-8405d367b722 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:59.986403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1ea5b-e889-4b00-baa0-8405d367b722 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:59.988891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 474/924] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:04:59.992867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5450-1c0d-4a1c-91d0-18940d210efc None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:59.993153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5450-1c0d-4a1c-91d0-18940d210efc None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:59.993428 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5450-1c0d-4a1c-91d0-18940d210efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:59.995188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d5d3d-c279-464a-b558-769a9cd77a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:59.995822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d5d3d-c279-464a-b558-769a9cd77a84 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:59.996103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d5d3d-c279-464a-b558-769a9cd77a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:59.996375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d5d3d-c279-464a-b558-769a9cd77a84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:59.997687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d5d3d-c279-464a-b558-769a9cd77a84 None admin] RBAC: Adding query filter params (interface=public, service_id=c437f4ff6aeb4f30a1cca47f03212186, region_id=RegionOne) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:59.997998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d5d3d-c279-464a-b558-769a9cd77a84 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:59.999433 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d5d3d-c279-464a-b558-769a9cd77a84 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:00.001021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:00.005796 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 451/925] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 7 16:04:59 2022] GET /identity/v3/endpoints?service_id=c437f4ff6aeb4f30a1cca47f03212186&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:00.011757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d5d3d-c279-464a-b558-769a9cd77a84 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:00.023642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-4c5e5450-1c0d-4a1c-91d0-18940d210efc None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:00.025148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4c5e5450-1c0d-4a1c-91d0-18940d210efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:00.045120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:00.123816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 475/926] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:04:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:05:00.124458 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d5d3d-c279-464a-b558-769a9cd77a84 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:00.162412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2a879-9f7b-43d8-8cd0-a492ffe8d6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:00.163022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2a879-9f7b-43d8-8cd0-a492ffe8d6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:00.163419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2a879-9f7b-43d8-8cd0-a492ffe8d6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:00.163697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2a879-9f7b-43d8-8cd0-a492ffe8d6d5 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:00.164330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2a879-9f7b-43d8-8cd0-a492ffe8d6d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:00.165689 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2a879-9f7b-43d8-8cd0-a492ffe8d6d5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:00.172852 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 452/927] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 7 16:05:00 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:05:00.177964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63224b39-a0a8-4935-b5c1-743c9944e96f None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:00.178245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63224b39-a0a8-4935-b5c1-743c9944e96f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:00.178515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63224b39-a0a8-4935-b5c1-743c9944e96f None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:00.180441 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 476/928] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:00.186235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942924a-5827-4a00-a90d-c4f9c995a394 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:00.186546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942924a-5827-4a00-a90d-c4f9c995a394 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:00.186792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942924a-5827-4a00-a90d-c4f9c995a394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:00.208161 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-2942924a-5827-4a00-a90d-c4f9c995a394 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:00.209382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2942924a-5827-4a00-a90d-c4f9c995a394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:00.274185 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:00.286067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 453/929] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:00.390816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63224b39-a0a8-4935-b5c1-743c9944e96f None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:00.498434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63224b39-a0a8-4935-b5c1-743c9944e96f None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:00.536413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04814e88-6dc1-4937-9b2d-a40c9be5b17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:00.537022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04814e88-6dc1-4937-9b2d-a40c9be5b17d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:00.537344 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04814e88-6dc1-4937-9b2d-a40c9be5b17d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:00.537620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04814e88-6dc1-4937-9b2d-a40c9be5b17d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:00.548327 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04814e88-6dc1-4937-9b2d-a40c9be5b17d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-04814e88-6dc1-4937-9b2d-a40c9be5b17d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:05:00.553278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:00.554218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:00.555536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 477/930] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 7 16:05:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:00.562671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942924a-5827-4a00-a90d-c4f9c995a394 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:00.570606 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04814e88-6dc1-4937-9b2d-a40c9be5b17d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:00.588126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:00.682597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04814e88-6dc1-4937-9b2d-a40c9be5b17d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:00.696696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942924a-5827-4a00-a90d-c4f9c995a394 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:00.718217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1533a3-177b-49a2-b787-c90b10a0fd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:00.719173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1533a3-177b-49a2-b787-c90b10a0fd89 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:00.719646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1533a3-177b-49a2-b787-c90b10a0fd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:00.720396 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1533a3-177b-49a2-b787-c90b10a0fd89 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:00.721590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1533a3-177b-49a2-b787-c90b10a0fd89 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:00.724033 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1533a3-177b-49a2-b787-c90b10a0fd89 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-ff1533a3-177b-49a2-b787-c90b10a0fd89 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:00.733878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:00.734838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:00.735730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 478/931] 10.222.0.15 () {56 vars in 1132 bytes} [Wed Sep 7 16:05:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:00.742584 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1533a3-177b-49a2-b787-c90b10a0fd89 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:00.742944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f04d1b-e085-456d-9655-d7de8c08ef6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:00.743711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f04d1b-e085-456d-9655-d7de8c08ef6b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:00.744276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f04d1b-e085-456d-9655-d7de8c08ef6b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:00.744649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f04d1b-e085-456d-9655-d7de8c08ef6b None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:00.745421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f04d1b-e085-456d-9655-d7de8c08ef6b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:00.745818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f04d1b-e085-456d-9655-d7de8c08ef6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:00.748358 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f04d1b-e085-456d-9655-d7de8c08ef6b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:00.762672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 454/932] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 7 16:05:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:00.764139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f04d1b-e085-456d-9655-d7de8c08ef6b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:00.825507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:00.849599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:00.874466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f04d1b-e085-456d-9655-d7de8c08ef6b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:00.881422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1533a3-177b-49a2-b787-c90b10a0fd89 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:00.912051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126729c-b810-47be-9976-6f02cc815f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:00.912842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126729c-b810-47be-9976-6f02cc815f81 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:00.913069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126729c-b810-47be-9976-6f02cc815f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:00.913351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126729c-b810-47be-9976-6f02cc815f81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:00.925756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d76585-7382-4e73-9545-09f48b3f6b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:00.926628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d76585-7382-4e73-9545-09f48b3f6b15 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:00.927068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d76585-7382-4e73-9545-09f48b3f6b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:00.927476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d76585-7382-4e73-9545-09f48b3f6b15 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:00.928703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d76585-7382-4e73-9545-09f48b3f6b15 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:00.929452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d76585-7382-4e73-9545-09f48b3f6b15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:00.930312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126729c-b810-47be-9976-6f02cc815f81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:00.931419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d76585-7382-4e73-9545-09f48b3f6b15 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:00.931874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2126729c-b810-47be-9976-6f02cc815f81 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:00.939816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 479/933] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 7 16:05:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:00.942689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d76585-7382-4e73-9545-09f48b3f6b15 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-2126729c-b810-47be-9976-6f02cc815f81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:00.958800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:00.959806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:05:00.960797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:00.960797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:00.960797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:00.960797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 455/934] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:00.966292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126729c-b810-47be-9976-6f02cc815f81 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:01.088831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d76585-7382-4e73-9545-09f48b3f6b15 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:01.099985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126729c-b810-47be-9976-6f02cc815f81 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:01.108368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:01.138183 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336a72c-cbf1-42b0-99c9-fb00094886e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:01.138765 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a6c5a-ab4c-4741-86f4-61b3e08c6d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:01.139047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336a72c-cbf1-42b0-99c9-fb00094886e6 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:01.139425 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336a72c-cbf1-42b0-99c9-fb00094886e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:01.139818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336a72c-cbf1-42b0-99c9-fb00094886e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:01.140063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a6c5a-ab4c-4741-86f4-61b3e08c6d7b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:01.140492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a6c5a-ab4c-4741-86f4-61b3e08c6d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:01.140749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336a72c-cbf1-42b0-99c9-fb00094886e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:01.141098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a6c5a-ab4c-4741-86f4-61b3e08c6d7b None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:01.141206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336a72c-cbf1-42b0-99c9-fb00094886e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:01.141942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a6c5a-ab4c-4741-86f4-61b3e08c6d7b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:01.142382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a6c5a-ab4c-4741-86f4-61b3e08c6d7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:01.142859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f336a72c-cbf1-42b0-99c9-fb00094886e6 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:01.144083 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a6c5a-ab4c-4741-86f4-61b3e08c6d7b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:01.155514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 480/935] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 7 16:05:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:01.156160 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a6c5a-ab4c-4741-86f4-61b3e08c6d7b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:01.161326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:01.163720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 456/936] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:01.169294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336a72c-cbf1-42b0-99c9-fb00094886e6 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:01.279278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f336a72c-cbf1-42b0-99c9-fb00094886e6 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:01.296805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a6c5a-ab4c-4741-86f4-61b3e08c6d7b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:01.323485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e0211-4ef9-4b19-9b84-0d9c2f39f96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:01.324175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e0211-4ef9-4b19-9b84-0d9c2f39f96d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:01.324768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e0211-4ef9-4b19-9b84-0d9c2f39f96d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:01.325687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e0211-4ef9-4b19-9b84-0d9c2f39f96d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:01.334345 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e0211-4ef9-4b19-9b84-0d9c2f39f96d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:01.335599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e0211-4ef9-4b19-9b84-0d9c2f39f96d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:01.344214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3081a0-d61c-4721-b6fe-5aca92bb7799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:01.345084 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3081a0-d61c-4721-b6fe-5aca92bb7799 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:01.345569 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3081a0-d61c-4721-b6fe-5aca92bb7799 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:01.346028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3081a0-d61c-4721-b6fe-5aca92bb7799 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:01.346888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3081a0-d61c-4721-b6fe-5aca92bb7799 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-a21e0211-4ef9-4b19-9b84-0d9c2f39f96d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:05:01.348108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:01.349045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:01.350024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:01.350024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:01.350024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:05:01.350024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:05:01.350024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:05:01.350024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:05:01.350024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:05:01.350024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:05:01.350024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:05:01.350024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:01.351815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3081a0-d61c-4721-b6fe-5aca92bb7799 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:01.353994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 457/937] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 7 16:05:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:01.355247 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e0211-4ef9-4b19-9b84-0d9c2f39f96d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:01.384362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:01.460225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 481/938] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:05:01 2022] POST /identity/v3/endpoints => generated 347 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:05:01.465475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e0211-4ef9-4b19-9b84-0d9c2f39f96d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:01.514661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90e4b2-02d6-424f-be9b-511391e9434e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:01.515271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90e4b2-02d6-424f-be9b-511391e9434e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:01.515548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90e4b2-02d6-424f-be9b-511391e9434e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:01.515970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90e4b2-02d6-424f-be9b-511391e9434e None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:01.516714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90e4b2-02d6-424f-be9b-511391e9434e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:01.517020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90e4b2-02d6-424f-be9b-511391e9434e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:01.518516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90e4b2-02d6-424f-be9b-511391e9434e None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:01.543141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 458/939] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 7 16:05:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:01.545653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3081a0-d61c-4721-b6fe-5aca92bb7799 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:01.615333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:01.661943 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3081a0-d61c-4721-b6fe-5aca92bb7799 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:01.702211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7a930f-bd8a-41c3-8d8e-11d304e107e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:01.702878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a930f-bd8a-41c3-8d8e-11d304e107e0 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:01.703156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a930f-bd8a-41c3-8d8e-11d304e107e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:01.703424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a930f-bd8a-41c3-8d8e-11d304e107e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:01.716746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a930f-bd8a-41c3-8d8e-11d304e107e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:01.718221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7a930f-bd8a-41c3-8d8e-11d304e107e0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-ef7a930f-bd8a-41c3-8d8e-11d304e107e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:01.733649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:01.734663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:05:01.735824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:01.735824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:01.735824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:01.735824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 482/940] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:01.742124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90e4b2-02d6-424f-be9b-511391e9434e None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:01.810333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10268af-b843-4823-ad0b-5edd2e84e500 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:01.810708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10268af-b843-4823-ad0b-5edd2e84e500 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:01.811057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10268af-b843-4823-ad0b-5edd2e84e500 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:01.812754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 483/941] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:05:01.817947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16ee82-e27d-4b18-8058-7f10e8ddd9fc None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:01.818127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16ee82-e27d-4b18-8058-7f10e8ddd9fc None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:01.818520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16ee82-e27d-4b18-8058-7f10e8ddd9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:01.837513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-cb16ee82-e27d-4b18-8058-7f10e8ddd9fc None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:01.838829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-cb16ee82-e27d-4b18-8058-7f10e8ddd9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:01.852157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:01.856339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90e4b2-02d6-424f-be9b-511391e9434e None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:01.901105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344564b-79a9-459f-90b6-8c2c34473be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:01.902986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344564b-79a9-459f-90b6-8c2c34473be6 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:01.903503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344564b-79a9-459f-90b6-8c2c34473be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:01.903991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344564b-79a9-459f-90b6-8c2c34473be6 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:01.904983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344564b-79a9-459f-90b6-8c2c34473be6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:01.905773 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344564b-79a9-459f-90b6-8c2c34473be6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:01.907609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344564b-79a9-459f-90b6-8c2c34473be6 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:01.916039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 484/942] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:05:01.920903 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 459/943] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:01.928656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16ee82-e27d-4b18-8058-7f10e8ddd9fc None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:01.971169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:02.030601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16ee82-e27d-4b18-8058-7f10e8ddd9fc None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:02.066622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5355b1-ecc7-4eba-9cec-00bb0f703ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:02.067261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5355b1-ecc7-4eba-9cec-00bb0f703ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:02.067539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5355b1-ecc7-4eba-9cec-00bb0f703ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:02.067830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5355b1-ecc7-4eba-9cec-00bb0f703ad3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:02.074672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5355b1-ecc7-4eba-9cec-00bb0f703ad3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:02.076155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5355b1-ecc7-4eba-9cec-00bb0f703ad3 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-6e5355b1-ecc7-4eba-9cec-00bb0f703ad3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:05:02.087263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:02.088192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:02.089514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:05:02.089514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:05:02.089514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:05:02.089514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:05:02.089514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:05:02.089514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:05:02.089514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:02.089514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:02.093071 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 485/944] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:05:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:02.094427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344564b-79a9-459f-90b6-8c2c34473be6 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:02.200984 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344564b-79a9-459f-90b6-8c2c34473be6 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:02.203235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5355b1-ecc7-4eba-9cec-00bb0f703ad3 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:02.240956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa97206-074d-4a08-ab47-14a233b399e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:02.241817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa97206-074d-4a08-ab47-14a233b399e3 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:02.242255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa97206-074d-4a08-ab47-14a233b399e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:02.242665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa97206-074d-4a08-ab47-14a233b399e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:02.243500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa97206-074d-4a08-ab47-14a233b399e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:02.243971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa97206-074d-4a08-ab47-14a233b399e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:02.245590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa97206-074d-4a08-ab47-14a233b399e3 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:02.256146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:02.259612 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 460/945] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:05:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:02.267679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa97206-074d-4a08-ab47-14a233b399e3 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:02.334471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5355b1-ecc7-4eba-9cec-00bb0f703ad3 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:02.336673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:02.377075 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e10cda7-a0ad-4905-abb3-91be58f4548e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:02.377694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10cda7-a0ad-4905-abb3-91be58f4548e None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:02.377972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10cda7-a0ad-4905-abb3-91be58f4548e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:02.378246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e10cda7-a0ad-4905-abb3-91be58f4548e None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:02.379038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10cda7-a0ad-4905-abb3-91be58f4548e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:02.381062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e10cda7-a0ad-4905-abb3-91be58f4548e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:02.384429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: WARNING keystone.server.flask.common [None req-2e10cda7-a0ad-4905-abb3-91be58f4548e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 07 16:05:02.389148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa97206-074d-4a08-ab47-14a233b399e3 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:02.426512 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ca0b7-70df-4a87-b807-cc2401d162eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:02.427170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ca0b7-70df-4a87-b807-cc2401d162eb None admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:02.427447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ca0b7-70df-4a87-b807-cc2401d162eb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:02.427715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ca0b7-70df-4a87-b807-cc2401d162eb None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/6b6893f49c96479c8f8dd11a372c257c/roles/2c8c5f15b3624270a26334ed3ebdf44a` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:02.430223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ca0b7-70df-4a87-b807-cc2401d162eb None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=6b6893f49c96479c8f8dd11a372c257c, role_id=2c8c5f15b3624270a26334ed3ebdf44a)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:02.431627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ca0b7-70df-4a87-b807-cc2401d162eb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:02.434481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.sql.core [None req-2e10cda7-a0ad-4905-abb3-91be58f4548e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 16:05:02.434481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 16:05:02.434481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [parameters: {'id': 'a1ca467adea44d70a30de009c1b88391', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 16:05:02.434481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81775) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-2e10cda7-a0ad-4905-abb3-91be58f4548e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a1ca467adea44d70a30de009c1b88391.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a1ca467adea44d70a30de009c1b88391. Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application conn.query(q) Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 16:05:02.442375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application result.read() Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 07 16:05:02.443872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application next(self.gen) Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application yield resource Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application next(self.gen) Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise self.value Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application session.commit() Sep 07 16:05:02.444837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self.session.flush() Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application compat.raise_( Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 16:05:02.445773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 16:05:02.446680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application conn.query(q) Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application result.read() Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 16:05:02.447506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [parameters: {'id': 'a1ca467adea44d70a30de009c1b88391', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:02.450239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:02.452371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a1ca467adea44d70a30de009c1b88391. Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 486/946] 10.222.0.15 () {60 vars in 1140 bytes} [Wed Sep 7 16:05:02 2022] POST /identity/v3/projects => generated 262 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a00cb5-4acb-4436-9570-34953a10c229 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:02.454553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a00cb5-4acb-4436-9570-34953a10c229 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:02.456110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a00cb5-4acb-4436-9570-34953a10c229 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:02.456110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 487/947] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:02.456110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a00cb5-4acb-4436-9570-34953a10c229 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:02.509649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 461/948] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:05:02 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/6b6893f49c96479c8f8dd11a372c257c/roles/2c8c5f15b3624270a26334ed3ebdf44a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:05:02.511004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5dfb0-31d0-4e6c-92e8-b242c769724a None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:02.511287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5dfb0-31d0-4e6c-92e8-b242c769724a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:02.511558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5dfb0-31d0-4e6c-92e8-b242c769724a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:02.534401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-40f5dfb0-31d0-4e6c-92e8-b242c769724a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:02.535728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-40f5dfb0-31d0-4e6c-92e8-b242c769724a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:02.586612 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a00cb5-4acb-4436-9570-34953a10c229 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:02.609892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:02.627924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 462/949] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:02.634772 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bfbd4-ca58-4886-9575-106f9944e85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:02.635855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bfbd4-ca58-4886-9575-106f9944e85e None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:02.636345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bfbd4-ca58-4886-9575-106f9944e85e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:02.636833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bfbd4-ca58-4886-9575-106f9944e85e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:02.646486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bfbd4-ca58-4886-9575-106f9944e85e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:02.648517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bfbd4-ca58-4886-9575-106f9944e85e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-f45bfbd4-ca58-4886-9575-106f9944e85e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:05:02.662289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:02.663242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:02.664176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:05:02.664176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:05:02.664176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:05:02.664176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:05:02.664176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:05:02.664176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:05:02.664176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:02.664176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:02.666666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 488/950] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:05:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:02.672602 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5dfb0-31d0-4e6c-92e8-b242c769724a None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:02.784105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5dfb0-31d0-4e6c-92e8-b242c769724a None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:02.820215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43242f06-83fc-4c8f-bd11-6d7763ff86ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:02.820823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43242f06-83fc-4c8f-bd11-6d7763ff86ee None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:02.821098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43242f06-83fc-4c8f-bd11-6d7763ff86ee None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:02.821366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43242f06-83fc-4c8f-bd11-6d7763ff86ee None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:02.822022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43242f06-83fc-4c8f-bd11-6d7763ff86ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:02.822319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43242f06-83fc-4c8f-bd11-6d7763ff86ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:02.823735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43242f06-83fc-4c8f-bd11-6d7763ff86ee None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:02.842712 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 463/951] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:05:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:02.902911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bfbd4-ca58-4886-9575-106f9944e85e None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:02.925890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:03.029184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bfbd4-ca58-4886-9575-106f9944e85e None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:03.068294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e5a93-e528-422c-a89f-235adda0fbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:03.068848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e5a93-e528-422c-a89f-235adda0fbed None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:03.069125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e5a93-e528-422c-a89f-235adda0fbed None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:03.069393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e5a93-e528-422c-a89f-235adda0fbed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:03.069985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e5a93-e528-422c-a89f-235adda0fbed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:03.071388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e5a93-e528-422c-a89f-235adda0fbed None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-935e5a93-e528-422c-a89f-235adda0fbed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:03.080853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:03.081951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:03.082848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 489/952] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 7 16:05:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:03.087825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43242f06-83fc-4c8f-bd11-6d7763ff86ee None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:03.145447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:03.197609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43242f06-83fc-4c8f-bd11-6d7763ff86ee None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:03.233643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9ac12-b7c1-4abb-874d-26be5d43ddba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:03.234409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9ac12-b7c1-4abb-874d-26be5d43ddba None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:03.234777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9ac12-b7c1-4abb-874d-26be5d43ddba None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:03.235140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9ac12-b7c1-4abb-874d-26be5d43ddba None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:03.236001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9ac12-b7c1-4abb-874d-26be5d43ddba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:03.236400 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9ac12-b7c1-4abb-874d-26be5d43ddba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:03.237869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9ac12-b7c1-4abb-874d-26be5d43ddba None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:03.243293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 464/953] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 7 16:05:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:03.249718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e5a93-e528-422c-a89f-235adda0fbed None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:03.341462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:03.360111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e5a93-e528-422c-a89f-235adda0fbed None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:03.396949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df5ffee-0d00-45df-b4ee-2796b6035ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:03.397568 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5ffee-0d00-45df-b4ee-2796b6035ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:03.397849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5ffee-0d00-45df-b4ee-2796b6035ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:03.398126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df5ffee-0d00-45df-b4ee-2796b6035ae9 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:03.398758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5ffee-0d00-45df-b4ee-2796b6035ae9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:03.399062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5ffee-0d00-45df-b4ee-2796b6035ae9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:03.400452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df5ffee-0d00-45df-b4ee-2796b6035ae9 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:03.405844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 490/954] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 7 16:05:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:03.411348 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9ac12-b7c1-4abb-874d-26be5d43ddba None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:03.522341 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9ac12-b7c1-4abb-874d-26be5d43ddba None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:03.536877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:03.561992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd971627-7b6f-4c56-bc2f-7f712b380fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:03.562596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd971627-7b6f-4c56-bc2f-7f712b380fe0 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:03.562876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd971627-7b6f-4c56-bc2f-7f712b380fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:03.563140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd971627-7b6f-4c56-bc2f-7f712b380fe0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:03.563836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd971627-7b6f-4c56-bc2f-7f712b380fe0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:03.565338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd971627-7b6f-4c56-bc2f-7f712b380fe0 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:03.653990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 465/955] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:05:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:05:03.773900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea5703-4ae9-4bf1-8018-81d087ffabe4 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:03.774345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea5703-4ae9-4bf1-8018-81d087ffabe4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:03.774719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea5703-4ae9-4bf1-8018-81d087ffabe4 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:03.776563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 491/956] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:03.782945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f696fb0-82ef-4b59-b34f-e13d15eb1241 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:03.783229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f696fb0-82ef-4b59-b34f-e13d15eb1241 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:03.783484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f696fb0-82ef-4b59-b34f-e13d15eb1241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:03.804309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0f696fb0-82ef-4b59-b34f-e13d15eb1241 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:03.805791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0f696fb0-82ef-4b59-b34f-e13d15eb1241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:03.885382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 466/957] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:04.151869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea5703-4ae9-4bf1-8018-81d087ffabe4 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:04.260685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea5703-4ae9-4bf1-8018-81d087ffabe4 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:04.297432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e208403-5e9f-44b3-b764-7fdb965cf6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:04.298520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e208403-5e9f-44b3-b764-7fdb965cf6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:04.299019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e208403-5e9f-44b3-b764-7fdb965cf6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:04.299484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e208403-5e9f-44b3-b764-7fdb965cf6b1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:04.300339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e208403-5e9f-44b3-b764-7fdb965cf6b1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:04.301896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e208403-5e9f-44b3-b764-7fdb965cf6b1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-4e208403-5e9f-44b3-b764-7fdb965cf6b1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:04.311776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:04.312816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 16:05:04.313674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:04.316456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 492/958] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:05:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:04.322496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f696fb0-82ef-4b59-b34f-e13d15eb1241 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:04.421698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:04.428643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f696fb0-82ef-4b59-b34f-e13d15eb1241 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:04.470859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32563e1-f9b4-4b82-a7fe-cbfec6005e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:04.471893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32563e1-f9b4-4b82-a7fe-cbfec6005e9e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:04.472395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32563e1-f9b4-4b82-a7fe-cbfec6005e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:04.472915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32563e1-f9b4-4b82-a7fe-cbfec6005e9e None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:04.473808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32563e1-f9b4-4b82-a7fe-cbfec6005e9e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:04.474324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32563e1-f9b4-4b82-a7fe-cbfec6005e9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:04.475909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32563e1-f9b4-4b82-a7fe-cbfec6005e9e None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:04.481357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 467/959] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:05:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:04.487434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e208403-5e9f-44b3-b764-7fdb965cf6b1 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:04.501426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:04.597394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e208403-5e9f-44b3-b764-7fdb965cf6b1 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:04.638919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17f9bf-a590-43ed-a4d4-c8041f9179fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:04.639922 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17f9bf-a590-43ed-a4d4-c8041f9179fe None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:04.640427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17f9bf-a590-43ed-a4d4-c8041f9179fe None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:04.640969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17f9bf-a590-43ed-a4d4-c8041f9179fe None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:04.641796 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17f9bf-a590-43ed-a4d4-c8041f9179fe None admin] RBAC: Adding query filter params (type=compute) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:04.642562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17f9bf-a590-43ed-a4d4-c8041f9179fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:04.644672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17f9bf-a590-43ed-a4d4-c8041f9179fe None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:04.650042 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 493/960] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:05:04 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:04.722086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b59876-e88e-416b-96b1-5435c80f1f17 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:04.722743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b59876-e88e-416b-96b1-5435c80f1f17 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:04.723240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b59876-e88e-416b-96b1-5435c80f1f17 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:04.725133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 468/961] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:04.730730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482c476-a209-4c27-ad5e-a2088a9db096 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:04.731062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482c476-a209-4c27-ad5e-a2088a9db096 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:04.731355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482c476-a209-4c27-ad5e-a2088a9db096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:04.751663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-2482c476-a209-4c27-ad5e-a2088a9db096 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:04.753257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2482c476-a209-4c27-ad5e-a2088a9db096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:04.840189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 494/962] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:05.120400 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b59876-e88e-416b-96b1-5435c80f1f17 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:05.237080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b59876-e88e-416b-96b1-5435c80f1f17 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:05.272069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ad221-9906-45b4-bd9a-13322365b98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:05.272621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ad221-9906-45b4-bd9a-13322365b98e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:05.272892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ad221-9906-45b4-bd9a-13322365b98e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:05.273159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ad221-9906-45b4-bd9a-13322365b98e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:05.283978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ad221-9906-45b4-bd9a-13322365b98e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-076ad221-9906-45b4-bd9a-13322365b98e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:05:05.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:05.289824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:05.290693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 469/963] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 7 16:05:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:05.297151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482c476-a209-4c27-ad5e-a2088a9db096 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:05.300321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:05.303108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb6a56-01e9-43bb-b46d-ad0df919159f None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:05.303395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb6a56-01e9-43bb-b46d-ad0df919159f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:05.303672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb6a56-01e9-43bb-b46d-ad0df919159f None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:05.306675 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 470/964] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:05.312183 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f522c-c726-4732-b902-ef7f344e20a2 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:05.312470 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f522c-c726-4732-b902-ef7f344e20a2 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:05.312750 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f522c-c726-4732-b902-ef7f344e20a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:05.333970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-711f522c-c726-4732-b902-ef7f344e20a2 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:05.335991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-711f522c-c726-4732-b902-ef7f344e20a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:05.380166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:05.422499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 471/965] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:05.437226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482c476-a209-4c27-ad5e-a2088a9db096 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:05.473315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bc8d0-5709-47c6-8f17-83513d4a6f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:05.474314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bc8d0-5709-47c6-8f17-83513d4a6f0c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:05.474795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bc8d0-5709-47c6-8f17-83513d4a6f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:05.475276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bc8d0-5709-47c6-8f17-83513d4a6f0c None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:05.476171 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bc8d0-5709-47c6-8f17-83513d4a6f0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:05.476692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bc8d0-5709-47c6-8f17-83513d4a6f0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:05.478284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4bc8d0-5709-47c6-8f17-83513d4a6f0c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:05.491881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 495/966] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 7 16:05:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:05.493953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f522c-c726-4732-b902-ef7f344e20a2 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:05.600143 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f522c-c726-4732-b902-ef7f344e20a2 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:05.635287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ea545-7efe-4446-ae04-21d1f78980a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:05.636266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ea545-7efe-4446-ae04-21d1f78980a3 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:05.636749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ea545-7efe-4446-ae04-21d1f78980a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:05.637759 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ea545-7efe-4446-ae04-21d1f78980a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:05.648385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ea545-7efe-4446-ae04-21d1f78980a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:05.650081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ea545-7efe-4446-ae04-21d1f78980a3 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-739ea545-7efe-4446-ae04-21d1f78980a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:05.666461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:05.667399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:05:05.668330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:05.668330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:05.668330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:05.673976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 472/967] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:05.676428 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bc8d0-5709-47c6-8f17-83513d4a6f0c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:05.744465 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:05.769422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ea545-7efe-4446-ae04-21d1f78980a3 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:05.776118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:05.791043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bc8d0-5709-47c6-8f17-83513d4a6f0c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:05.838153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbecb7-1123-4851-b7a7-b49814b42cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:05.839681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbecb7-1123-4851-b7a7-b49814b42cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:05.841090 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbecb7-1123-4851-b7a7-b49814b42cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:05.841576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbecb7-1123-4851-b7a7-b49814b42cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:05.842666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbecb7-1123-4851-b7a7-b49814b42cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:05.843557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbecb7-1123-4851-b7a7-b49814b42cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:05.845598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbecb7-1123-4851-b7a7-b49814b42cd7 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:05.866711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 496/968] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:05.869088 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbecb7-1123-4851-b7a7-b49814b42cd7 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:05.895359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ea545-7efe-4446-ae04-21d1f78980a3 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:05.932894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c4d98-6fa1-4c38-8a4d-0d49870f9c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:05.933516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4d98-6fa1-4c38-8a4d-0d49870f9c83 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:05.933806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4d98-6fa1-4c38-8a4d-0d49870f9c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:05.934111 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c4d98-6fa1-4c38-8a4d-0d49870f9c83 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:05.934757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c4d98-6fa1-4c38-8a4d-0d49870f9c83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:05.936223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70c4d98-6fa1-4c38-8a4d-0d49870f9c83 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:05.938817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: WARNING keystone.server.flask.common [None req-f70c4d98-6fa1-4c38-8a4d-0d49870f9c83 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 07 16:05:05.973878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbecb7-1123-4851-b7a7-b49814b42cd7 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:05.988837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.sql.core [None req-f70c4d98-6fa1-4c38-8a4d-0d49870f9c83 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 16:05:05.988837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 16:05:05.988837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [parameters: {'user_id': '3a259485668f47479650f994ae1a84d1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 16:05:05.988837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81776) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-f70c4d98-6fa1-4c38-8a4d-0d49870f9c83 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application conn.query(q) Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application result.read() Sep 07 16:05:06.006180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 16:05:06.007140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application next(self.gen) Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application yield resource Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application next(self.gen) Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise self.value Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application session.commit() Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 16:05:06.008617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self.session.flush() Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application compat.raise_( Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 16:05:06.009541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 16:05:06.010432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application conn.query(q) Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application result.read() Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 16:05:06.011804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [parameters: {'user_id': '3a259485668f47479650f994ae1a84d1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:06.014188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 07 16:05:06.015128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 07 16:05:06.016080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 07 16:05:06.016080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 07 16:05:06.016080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 16:05:06.016080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 16:05:06.016080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 16:05:06.016080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:06.016080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 473/969] 10.222.0.15 () {60 vars in 1131 bytes} [Wed Sep 7 16:05:05 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:06.016547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:06.016620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c224f-5554-4d56-9b58-ef13668068fa None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.016620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c224f-5554-4d56-9b58-ef13668068fa None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.016620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c224f-5554-4d56-9b58-ef13668068fa None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.016620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 474/970] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:06.016855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c224f-5554-4d56-9b58-ef13668068fa None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:06.028447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b3125-8f5a-45bd-bdb2-08a316092eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:06.029471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3125-8f5a-45bd-bdb2-08a316092eca None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.029750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3125-8f5a-45bd-bdb2-08a316092eca None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.030022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b3125-8f5a-45bd-bdb2-08a316092eca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.037971 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b3125-8f5a-45bd-bdb2-08a316092eca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:06.039430 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8b3125-8f5a-45bd-bdb2-08a316092eca None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-2b8b3125-8f5a-45bd-bdb2-08a316092eca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:05:06.052685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:06.053807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:06.055020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:06.055020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:06.055020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:05:06.055020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:05:06.055020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:05:06.055020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:05:06.055020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:05:06.055020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:05:06.055020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 16:05:06.055020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:06.055020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 497/971] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 7 16:05:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:06.055639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f4c2b-3a3a-4ad0-ac1f-fe50013c655b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.055639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f4c2b-3a3a-4ad0-ac1f-fe50013c655b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.056439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f4c2b-3a3a-4ad0-ac1f-fe50013c655b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.074213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e85f4c2b-3a3a-4ad0-ac1f-fe50013c655b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:06.075987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e85f4c2b-3a3a-4ad0-ac1f-fe50013c655b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:06.089054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:06.129400 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c224f-5554-4d56-9b58-ef13668068fa None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:06.147016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 498/972] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:06.148151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f4c2b-3a3a-4ad0-ac1f-fe50013c655b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:06.176763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2d834-4517-44a9-ad51-36c7c1fdfda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:06.177497 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2d834-4517-44a9-ad51-36c7c1fdfda2 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.178113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2d834-4517-44a9-ad51-36c7c1fdfda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.178479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2d834-4517-44a9-ad51-36c7c1fdfda2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.190656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2d834-4517-44a9-ad51-36c7c1fdfda2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:06.193226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2d834-4517-44a9-ad51-36c7c1fdfda2 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-74f2d834-4517-44a9-ad51-36c7c1fdfda2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:05:06.208578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:06.209543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:06.210501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:06.210501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:06.210501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:05:06.210501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:05:06.210501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:05:06.210501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:05:06.210501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:05:06.210501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:05:06.210501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 16:05:06.210501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:06.214712 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 475/973] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:05:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:06.216099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2d834-4517-44a9-ad51-36c7c1fdfda2 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:06.277998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f4c2b-3a3a-4ad0-ac1f-fe50013c655b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:06.301243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:06.321067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbbe9e-0073-4da4-b8a6-9158d5ad9435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:06.321998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbbe9e-0073-4da4-b8a6-9158d5ad9435 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.322488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbbe9e-0073-4da4-b8a6-9158d5ad9435 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.322977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbbe9e-0073-4da4-b8a6-9158d5ad9435 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.323947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbbe9e-0073-4da4-b8a6-9158d5ad9435 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:06.324700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbbe9e-0073-4da4-b8a6-9158d5ad9435 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:06.326848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbbe9e-0073-4da4-b8a6-9158d5ad9435 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:06.346390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 499/974] 10.222.0.15 () {56 vars in 1149 bytes} [Wed Sep 7 16:05:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:06.355829 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2d834-4517-44a9-ad51-36c7c1fdfda2 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:06.356337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbbe9e-0073-4da4-b8a6-9158d5ad9435 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:06.379619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:06.395363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883485dd-c170-488b-bfd7-5dae980d58ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:06.396003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883485dd-c170-488b-bfd7-5dae980d58ba None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.398661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883485dd-c170-488b-bfd7-5dae980d58ba None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.398661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883485dd-c170-488b-bfd7-5dae980d58ba None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.398661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883485dd-c170-488b-bfd7-5dae980d58ba None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:06.398661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883485dd-c170-488b-bfd7-5dae980d58ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:06.400624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883485dd-c170-488b-bfd7-5dae980d58ba None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:06.447312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 476/975] 10.222.0.15 () {56 vars in 1101 bytes} [Wed Sep 7 16:05:06 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:06.450544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883485dd-c170-488b-bfd7-5dae980d58ba None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:06.461481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbbe9e-0073-4da4-b8a6-9158d5ad9435 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:06.502873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ba099-3d89-4812-adbd-d7be26a99e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:06.503824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ba099-3d89-4812-adbd-d7be26a99e31 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.504210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ba099-3d89-4812-adbd-d7be26a99e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.504586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ba099-3d89-4812-adbd-d7be26a99e31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.560215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:06.580440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883485dd-c170-488b-bfd7-5dae980d58ba None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:06.587650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ba099-3d89-4812-adbd-d7be26a99e31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:06.589444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ba099-3d89-4812-adbd-d7be26a99e31 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:06.622767 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626cea5-a289-4931-b0bc-8103460f0f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:06.623431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626cea5-a289-4931-b0bc-8103460f0f08 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.623697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626cea5-a289-4931-b0bc-8103460f0f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.623994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626cea5-a289-4931-b0bc-8103460f0f08 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.624591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626cea5-a289-4931-b0bc-8103460f0f08 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:06.625976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626cea5-a289-4931-b0bc-8103460f0f08 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-3626cea5-a289-4931-b0bc-8103460f0f08 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:06.636256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:06.637347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:06.638208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 477/976] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 7 16:05:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:06.645554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626cea5-a289-4931-b0bc-8103460f0f08 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:06.706446 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:06.726034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 500/977] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:05:06 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:05:06.734312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ba099-3d89-4812-adbd-d7be26a99e31 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:06.778595 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626cea5-a289-4931-b0bc-8103460f0f08 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:06.819118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38d930-eb88-413e-89a9-550c887d0b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:06.819735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38d930-eb88-413e-89a9-550c887d0b41 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.820098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38d930-eb88-413e-89a9-550c887d0b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.820387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38d930-eb88-413e-89a9-550c887d0b41 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.821056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38d930-eb88-413e-89a9-550c887d0b41 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:06.821363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38d930-eb88-413e-89a9-550c887d0b41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:06.822737 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38d930-eb88-413e-89a9-550c887d0b41 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:06.827200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:06.827550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 478/978] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 7 16:05:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:06.833123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38d930-eb88-413e-89a9-550c887d0b41 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:06.856695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ba099-3d89-4812-adbd-d7be26a99e31 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:06.897823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46974d8-dce3-4fbf-8528-766aef8bbedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:06.898375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46974d8-dce3-4fbf-8528-766aef8bbedb None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.898676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46974d8-dce3-4fbf-8528-766aef8bbedb None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.898923 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46974d8-dce3-4fbf-8528-766aef8bbedb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.910859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46974d8-dce3-4fbf-8528-766aef8bbedb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:06.912361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46974d8-dce3-4fbf-8528-766aef8bbedb None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-b46974d8-dce3-4fbf-8528-766aef8bbedb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:06.932875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:06.933915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:05:06.940602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:06.940602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:06.940602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:06.940602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 501/979] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:06.940602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf852fbc-0b7f-4648-883b-6b010bf42c0f None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.940602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf852fbc-0b7f-4648-883b-6b010bf42c0f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.940602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf852fbc-0b7f-4648-883b-6b010bf42c0f None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.940602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 502/980] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:06.940602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf852fbc-0b7f-4648-883b-6b010bf42c0f None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:06.944011 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38d930-eb88-413e-89a9-550c887d0b41 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:06.963369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:06.987199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d9f87-6709-4b2b-813d-fb237954f08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:06.987912 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d9f87-6709-4b2b-813d-fb237954f08b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.988242 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d9f87-6709-4b2b-813d-fb237954f08b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.988568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d9f87-6709-4b2b-813d-fb237954f08b None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:06.989247 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d9f87-6709-4b2b-813d-fb237954f08b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:06.989551 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d9f87-6709-4b2b-813d-fb237954f08b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:06.990892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d9f87-6709-4b2b-813d-fb237954f08b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:06.995722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 479/981] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 7 16:05:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:06.996979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f694366-bc49-43f6-a728-421ef34815cd None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:06.997264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f694366-bc49-43f6-a728-421ef34815cd None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:06.997534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f694366-bc49-43f6-a728-421ef34815cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:07.015970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-2f694366-bc49-43f6-a728-421ef34815cd None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:07.017520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2f694366-bc49-43f6-a728-421ef34815cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:07.049773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf852fbc-0b7f-4648-883b-6b010bf42c0f None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:07.073984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:07.081975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5838f61-ecf4-4e12-8b03-803f393970fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:07.082111 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 480/982] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:07.082579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5838f61-ecf4-4e12-8b03-803f393970fe None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:07.084643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5838f61-ecf4-4e12-8b03-803f393970fe None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:07.084643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5838f61-ecf4-4e12-8b03-803f393970fe None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:07.084643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5838f61-ecf4-4e12-8b03-803f393970fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:07.084643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5838f61-ecf4-4e12-8b03-803f393970fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:07.085442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5838f61-ecf4-4e12-8b03-803f393970fe None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:07.088079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f694366-bc49-43f6-a728-421ef34815cd None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:07.097620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 503/983] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:07.102946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5838f61-ecf4-4e12-8b03-803f393970fe None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:07.203973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f694366-bc49-43f6-a728-421ef34815cd None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:07.205103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5838f61-ecf4-4e12-8b03-803f393970fe None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:07.206979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:07.243145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf8f1c-b057-4277-8b6d-62cd9c8a96fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:07.243844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf8f1c-b057-4277-8b6d-62cd9c8a96fb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:07.244130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf8f1c-b057-4277-8b6d-62cd9c8a96fb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:07.244390 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf8f1c-b057-4277-8b6d-62cd9c8a96fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:07.245109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf8f1c-b057-4277-8b6d-62cd9c8a96fb None admin] RBAC: Adding query filter params (interface=admin, service_id=c437f4ff6aeb4f30a1cca47f03212186, region_id=RegionOne) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:07.245413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf8f1c-b057-4277-8b6d-62cd9c8a96fb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:07.246794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf8f1c-b057-4277-8b6d-62cd9c8a96fb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:07.252088 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 481/984] 10.222.0.15 () {56 vars in 1249 bytes} [Wed Sep 7 16:05:07 2022] GET /identity/v3/endpoints?service_id=c437f4ff6aeb4f30a1cca47f03212186&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:07.254031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b1bb3-1911-4805-91a0-2cc54f834e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:07.254666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b1bb3-1911-4805-91a0-2cc54f834e30 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:07.254946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b1bb3-1911-4805-91a0-2cc54f834e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:07.255218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b1bb3-1911-4805-91a0-2cc54f834e30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:07.257894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf8f1c-b057-4277-8b6d-62cd9c8a96fb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:07.264053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b1bb3-1911-4805-91a0-2cc54f834e30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:07.265488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b1bb3-1911-4805-91a0-2cc54f834e30 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-bd7b1bb3-1911-4805-91a0-2cc54f834e30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:05:07.277279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:07.278978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:07.280729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:05:07.280729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:05:07.280729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:05:07.280729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:05:07.280729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:05:07.280729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:05:07.280729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:07.280729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:07.284017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 504/985] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:05:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:07.284567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b1bb3-1911-4805-91a0-2cc54f834e30 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:07.334737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:07.363102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf8f1c-b057-4277-8b6d-62cd9c8a96fb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:07.390413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b1bb3-1911-4805-91a0-2cc54f834e30 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:07.398656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180d97d-3929-4ee7-896b-9d1e7c46b56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:07.399264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d97d-3929-4ee7-896b-9d1e7c46b56f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:07.399540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d97d-3929-4ee7-896b-9d1e7c46b56f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:07.399844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9180d97d-3929-4ee7-896b-9d1e7c46b56f None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:07.400450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180d97d-3929-4ee7-896b-9d1e7c46b56f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:07.401807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9180d97d-3929-4ee7-896b-9d1e7c46b56f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:07.408387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 482/986] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 7 16:05:07 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:05:07.410348 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180d97d-3929-4ee7-896b-9d1e7c46b56f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:07.424389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb64048-dd4e-410b-ba7d-e28e52b9cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:07.425026 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64048-dd4e-410b-ba7d-e28e52b9cfce None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:07.425307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64048-dd4e-410b-ba7d-e28e52b9cfce None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:07.425583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb64048-dd4e-410b-ba7d-e28e52b9cfce None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:07.426293 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb64048-dd4e-410b-ba7d-e28e52b9cfce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:07.426600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb64048-dd4e-410b-ba7d-e28e52b9cfce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:07.428125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb64048-dd4e-410b-ba7d-e28e52b9cfce None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:07.442451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 505/987] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:05:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:07.448658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb64048-dd4e-410b-ba7d-e28e52b9cfce None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:07.453302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:07.530451 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9180d97d-3929-4ee7-896b-9d1e7c46b56f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:07.565680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb64048-dd4e-410b-ba7d-e28e52b9cfce None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:07.569411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f6063-4d33-4772-9bc7-fa7edeba7959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:07.570037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f6063-4d33-4772-9bc7-fa7edeba7959 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:07.570320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f6063-4d33-4772-9bc7-fa7edeba7959 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:07.570594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f6063-4d33-4772-9bc7-fa7edeba7959 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:07.571283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f6063-4d33-4772-9bc7-fa7edeba7959 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:07.572694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f6063-4d33-4772-9bc7-fa7edeba7959 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:07.575317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:07.602256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753606f2-b822-4f38-b397-c3889ad9c9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:07.602856 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753606f2-b822-4f38-b397-c3889ad9c9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:07.603174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753606f2-b822-4f38-b397-c3889ad9c9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:07.603452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753606f2-b822-4f38-b397-c3889ad9c9e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:07.604192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753606f2-b822-4f38-b397-c3889ad9c9e7 None admin] RBAC: Adding query filter params (role.id=2c8c5f15b3624270a26334ed3ebdf44a, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=6b6893f49c96479c8f8dd11a372c257c) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:07.604498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753606f2-b822-4f38-b397-c3889ad9c9e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:07.605952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753606f2-b822-4f38-b397-c3889ad9c9e7 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:07.611758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 506/988] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:05:07 2022] GET /identity/v3/role_assignments?user.id=6b6893f49c96479c8f8dd11a372c257c&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=2c8c5f15b3624270a26334ed3ebdf44a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:07.686014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 483/989] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:05:07 2022] POST /identity/v3/services => generated 242 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:05:09.601853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caab972-03bc-407c-b4a3-3361855c52ea None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:09.602376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caab972-03bc-407c-b4a3-3361855c52ea None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:09.602376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caab972-03bc-407c-b4a3-3361855c52ea None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:09.603783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 507/990] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:09.608954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832348b-b6b3-432f-9ae7-5de2dcaa7cae None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:09.609362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832348b-b6b3-432f-9ae7-5de2dcaa7cae None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:09.609528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832348b-b6b3-432f-9ae7-5de2dcaa7cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:09.627656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-6832348b-b6b3-432f-9ae7-5de2dcaa7cae None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:09.628873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6832348b-b6b3-432f-9ae7-5de2dcaa7cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:09.708162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 484/991] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:09.798628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749f2f4-cca4-40f2-9d03-e1c4f24183ba None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:09.798930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749f2f4-cca4-40f2-9d03-e1c4f24183ba None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:09.799178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749f2f4-cca4-40f2-9d03-e1c4f24183ba None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:09.800671 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 508/992] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:09.806049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979e9af-cbb7-448b-8573-90bc83b6705e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:09.806337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979e9af-cbb7-448b-8573-90bc83b6705e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:09.806615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979e9af-cbb7-448b-8573-90bc83b6705e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:09.825733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-6979e9af-cbb7-448b-8573-90bc83b6705e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:09.826924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6979e9af-cbb7-448b-8573-90bc83b6705e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:09.840350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:09.861802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27f934-b875-40a5-93f9-47b96f280545 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:09.861802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27f934-b875-40a5-93f9-47b96f280545 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:09.861802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27f934-b875-40a5-93f9-47b96f280545 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:09.861802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 509/993] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:05:09.861802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cf6eb-c90b-491e-8803-361148127de7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:09.861802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cf6eb-c90b-491e-8803-361148127de7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:09.862552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5cf6eb-c90b-491e-8803-361148127de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:09.879366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ad5cf6eb-c90b-491e-8803-361148127de7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:09.880581 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ad5cf6eb-c90b-491e-8803-361148127de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:09.900860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 485/994] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:09.951320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 510/995] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:05:09.973370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979e9af-cbb7-448b-8573-90bc83b6705e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:10.089588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979e9af-cbb7-448b-8573-90bc83b6705e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:10.123531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb2cfb-f974-43a6-ac33-df9266bc742b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:10.124105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2cfb-f974-43a6-ac33-df9266bc742b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:10.124492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2cfb-f974-43a6-ac33-df9266bc742b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:10.124772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb2cfb-f974-43a6-ac33-df9266bc742b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:10.125354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb2cfb-f974-43a6-ac33-df9266bc742b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:10.126716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb2cfb-f974-43a6-ac33-df9266bc742b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-63fb2cfb-f974-43a6-ac33-df9266bc742b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:10.135383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:10.140004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:10.141038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 486/996] 10.222.0.15 () {56 vars in 1126 bytes} [Wed Sep 7 16:05:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:10.143401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cf6eb-c90b-491e-8803-361148127de7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:10.175535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb2cfb-f974-43a6-ac33-df9266bc742b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:10.193075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:10.243922 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:10.247338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5cf6eb-c90b-491e-8803-361148127de7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:10.279748 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb2cfb-f974-43a6-ac33-df9266bc742b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:10.279924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e72aa3-99f0-485e-be4e-1e93bcdcb356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:10.280537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e72aa3-99f0-485e-be4e-1e93bcdcb356 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:10.280817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e72aa3-99f0-485e-be4e-1e93bcdcb356 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:10.281095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e72aa3-99f0-485e-be4e-1e93bcdcb356 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:10.281722 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e72aa3-99f0-485e-be4e-1e93bcdcb356 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:10.282023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e72aa3-99f0-485e-be4e-1e93bcdcb356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:10.283402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e72aa3-99f0-485e-be4e-1e93bcdcb356 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:10.288559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 511/997] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 7 16:05:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:10.289325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e72aa3-99f0-485e-be4e-1e93bcdcb356 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:10.320096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2abbf1-d095-4af5-bb35-eff6b829df5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:10.320729 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2abbf1-d095-4af5-bb35-eff6b829df5b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:10.321008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2abbf1-d095-4af5-bb35-eff6b829df5b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:10.321283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2abbf1-d095-4af5-bb35-eff6b829df5b None admin] PATH_INFO: `/v3/services/image` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:10.321863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2abbf1-d095-4af5-bb35-eff6b829df5b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:10.323229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2abbf1-d095-4af5-bb35-eff6b829df5b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-4f2abbf1-d095-4af5-bb35-eff6b829df5b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:10.331869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:10.332922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:10.334401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 487/998] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Sep 7 16:05:10 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:10.335202 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2abbf1-d095-4af5-bb35-eff6b829df5b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:10.389786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e72aa3-99f0-485e-be4e-1e93bcdcb356 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:10.409053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:10.423673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0b54e-8064-4b89-b152-3814743f7c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:10.424329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0b54e-8064-4b89-b152-3814743f7c7b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:10.424608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0b54e-8064-4b89-b152-3814743f7c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:10.432151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0b54e-8064-4b89-b152-3814743f7c7b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:10.432151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0b54e-8064-4b89-b152-3814743f7c7b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:10.432151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0b54e-8064-4b89-b152-3814743f7c7b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:10.432456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2abbf1-d095-4af5-bb35-eff6b829df5b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-bca0b54e-8064-4b89-b152-3814743f7c7b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:10.436386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:10.437628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 16:05:10.438855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:10.439911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 512/999] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:05:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:10.440857 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0b54e-8064-4b89-b152-3814743f7c7b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:10.465526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ae42c-fee1-4eb3-9cc9-47152bbfe49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:10.466133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ae42c-fee1-4eb3-9cc9-47152bbfe49b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:10.466412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ae42c-fee1-4eb3-9cc9-47152bbfe49b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:10.466687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ae42c-fee1-4eb3-9cc9-47152bbfe49b None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:10.467312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ae42c-fee1-4eb3-9cc9-47152bbfe49b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:10.467617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ae42c-fee1-4eb3-9cc9-47152bbfe49b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:10.468543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:10.469007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ae42c-fee1-4eb3-9cc9-47152bbfe49b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:10.473805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 488/1000] 10.222.0.15 () {56 vars in 1122 bytes} [Wed Sep 7 16:05:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:10.473805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ae42c-fee1-4eb3-9cc9-47152bbfe49b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:10.539937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0b54e-8064-4b89-b152-3814743f7c7b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:10.573599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f7d4f-930d-4651-aebe-0596ed477ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:10.574202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f7d4f-930d-4651-aebe-0596ed477ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:10.574525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f7d4f-930d-4651-aebe-0596ed477ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:10.574801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f7d4f-930d-4651-aebe-0596ed477ff6 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:10.575425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f7d4f-930d-4651-aebe-0596ed477ff6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:10.575728 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f7d4f-930d-4651-aebe-0596ed477ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:10.576396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ae42c-fee1-4eb3-9cc9-47152bbfe49b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:10.577119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f7d4f-930d-4651-aebe-0596ed477ff6 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:10.581769 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 513/1001] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:10.582516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f7d4f-930d-4651-aebe-0596ed477ff6 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:10.610940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfafe1e-3f30-4f53-aea9-0d5523dfa48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:10.611549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfafe1e-3f30-4f53-aea9-0d5523dfa48c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:10.611855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfafe1e-3f30-4f53-aea9-0d5523dfa48c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:10.612137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfafe1e-3f30-4f53-aea9-0d5523dfa48c None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:10.612760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfafe1e-3f30-4f53-aea9-0d5523dfa48c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:10.613068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfafe1e-3f30-4f53-aea9-0d5523dfa48c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:10.614426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfafe1e-3f30-4f53-aea9-0d5523dfa48c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:10.619369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 489/1002] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:05:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:10.620023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfafe1e-3f30-4f53-aea9-0d5523dfa48c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:10.627798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:10.674421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:10.682852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f7d4f-930d-4651-aebe-0596ed477ff6 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:10.716188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed76cc-ab84-4b28-9894-c068ca7b760f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:10.716789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed76cc-ab84-4b28-9894-c068ca7b760f None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:10.717066 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed76cc-ab84-4b28-9894-c068ca7b760f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:10.717338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed76cc-ab84-4b28-9894-c068ca7b760f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:10.717995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed76cc-ab84-4b28-9894-c068ca7b760f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:10.719353 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed76cc-ab84-4b28-9894-c068ca7b760f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:10.723035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfafe1e-3f30-4f53-aea9-0d5523dfa48c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:10.756193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a84f3a-8894-4df3-b71b-b87142e1c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:10.756785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a84f3a-8894-4df3-b71b-b87142e1c515 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:10.757058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a84f3a-8894-4df3-b71b-b87142e1c515 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:10.757325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a84f3a-8894-4df3-b71b-b87142e1c515 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:10.757940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a84f3a-8894-4df3-b71b-b87142e1c515 None admin] RBAC: Adding query filter params (type=image) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:10.758242 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a84f3a-8894-4df3-b71b-b87142e1c515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:10.759597 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a84f3a-8894-4df3-b71b-b87142e1c515 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:10.763904 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 490/1003] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:10.765829 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a84f3a-8894-4df3-b71b-b87142e1c515 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:10.827212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 514/1004] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:05:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:05:10.841096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:10.883278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a84f3a-8894-4df3-b71b-b87142e1c515 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:10.931549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c96be4-93ad-4741-86b6-635f88c88db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:10.932214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c96be4-93ad-4741-86b6-635f88c88db2 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:10.932557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c96be4-93ad-4741-86b6-635f88c88db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:10.932893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c96be4-93ad-4741-86b6-635f88c88db2 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:10.933630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c96be4-93ad-4741-86b6-635f88c88db2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:10.934038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c96be4-93ad-4741-86b6-635f88c88db2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:10.935625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c96be4-93ad-4741-86b6-635f88c88db2 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:10.946488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 491/1005] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:05:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:10.947649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed76cc-ab84-4b28-9894-c068ca7b760f None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:11.077131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed76cc-ab84-4b28-9894-c068ca7b760f None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:11.114849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd29d2-7662-4c06-8a43-6920f13806d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:11.115465 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd29d2-7662-4c06-8a43-6920f13806d8 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:11.115744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd29d2-7662-4c06-8a43-6920f13806d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:11.116046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd29d2-7662-4c06-8a43-6920f13806d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:11.116750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd29d2-7662-4c06-8a43-6920f13806d8 None admin] RBAC: Adding query filter params (interface=public, service_id=d369a4dd6eb343fbb7ec862b16ff71f7, region_id=RegionOne) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:11.117058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd29d2-7662-4c06-8a43-6920f13806d8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:11.118417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd29d2-7662-4c06-8a43-6920f13806d8 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:11.127808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 515/1006] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 7 16:05:10 2022] GET /identity/v3/endpoints?service_id=d369a4dd6eb343fbb7ec862b16ff71f7&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:11.129622 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c96be4-93ad-4741-86b6-635f88c88db2 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:11.232137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c96be4-93ad-4741-86b6-635f88c88db2 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:11.258574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:11.267413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9227a9a-c508-4125-bbaa-f478860f88e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:11.268055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9227a9a-c508-4125-bbaa-f478860f88e5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:11.268335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9227a9a-c508-4125-bbaa-f478860f88e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:11.268607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9227a9a-c508-4125-bbaa-f478860f88e5 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:11.269207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9227a9a-c508-4125-bbaa-f478860f88e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:11.270595 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9227a9a-c508-4125-bbaa-f478860f88e5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:11.279008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 492/1007] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 7 16:05:11 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:05:13.059918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855e8b8-d0f3-4608-a8d1-984279867335 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:13.060424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855e8b8-d0f3-4608-a8d1-984279867335 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:13.060531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855e8b8-d0f3-4608-a8d1-984279867335 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:13.061830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 516/1008] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:13.067187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cdc03-1c32-4837-a809-76098b712785 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:13.067601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cdc03-1c32-4837-a809-76098b712785 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:13.067825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cdc03-1c32-4837-a809-76098b712785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:13.082818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779c51-8814-4d5b-9e0f-3d4c57b0d3c3 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:13.083144 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779c51-8814-4d5b-9e0f-3d4c57b0d3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:13.083379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779c51-8814-4d5b-9e0f-3d4c57b0d3c3 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:13.084891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 517/1009] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:05:13.088260 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-8f8cdc03-1c32-4837-a809-76098b712785 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:13.089541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8f8cdc03-1c32-4837-a809-76098b712785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:13.089824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efd8b7-898e-416e-ae46-583fdb1e3f67 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:13.090112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efd8b7-898e-416e-ae46-583fdb1e3f67 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:13.090393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93efd8b7-898e-416e-ae46-583fdb1e3f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:13.109345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-93efd8b7-898e-416e-ae46-583fdb1e3f67 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:13.110662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-93efd8b7-898e-416e-ae46-583fdb1e3f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:13.125981 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:13.166688 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 493/1010] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:13.187591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 518/1011] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:05:13.441595 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cdc03-1c32-4837-a809-76098b712785 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:13.456066 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efd8b7-898e-416e-ae46-583fdb1e3f67 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:13.541734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cdc03-1c32-4837-a809-76098b712785 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:13.554142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93efd8b7-898e-416e-ae46-583fdb1e3f67 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:13.564360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:13.576027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b813b8a-88b7-4bdd-998c-f6d437eeb662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:13.576649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813b8a-88b7-4bdd-998c-f6d437eeb662 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:13.576939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813b8a-88b7-4bdd-998c-f6d437eeb662 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:13.577198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b813b8a-88b7-4bdd-998c-f6d437eeb662 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:13.578284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b813b8a-88b7-4bdd-998c-f6d437eeb662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:13.579738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b813b8a-88b7-4bdd-998c-f6d437eeb662 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:13.581313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 494/1012] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:13.582412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e493a-a280-4c41-8e3f-290ed4d14343 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:13.582680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e493a-a280-4c41-8e3f-290ed4d14343 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:13.582943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e493a-a280-4c41-8e3f-290ed4d14343 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:13.584282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 495/1013] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:13.587155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e493a-a280-4c41-8e3f-290ed4d14343 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:13.590895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0054cc7-2d08-4722-bcbf-72a89a5c8328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:13.591507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0054cc7-2d08-4722-bcbf-72a89a5c8328 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:13.591812 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0054cc7-2d08-4722-bcbf-72a89a5c8328 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:13.592098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0054cc7-2d08-4722-bcbf-72a89a5c8328 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:13.592860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0054cc7-2d08-4722-bcbf-72a89a5c8328 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:13.594216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0054cc7-2d08-4722-bcbf-72a89a5c8328 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:13.668818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 519/1014] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:05:13 2022] POST /identity/v3/services => generated 242 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:05:13.670017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636c25f-5325-4474-8f64-b693c55eacfb None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:13.670301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636c25f-5325-4474-8f64-b693c55eacfb None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:13.670585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636c25f-5325-4474-8f64-b693c55eacfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:13.685779 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:13.692423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-9636c25f-5325-4474-8f64-b693c55eacfb None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:13.693767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9636c25f-5325-4474-8f64-b693c55eacfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:13.703020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e493a-a280-4c41-8e3f-290ed4d14343 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:13.748109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30453d4-a400-4ce4-8df2-1d8552019086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:13.749180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30453d4-a400-4ce4-8df2-1d8552019086 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:13.749663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30453d4-a400-4ce4-8df2-1d8552019086 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:13.749942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30453d4-a400-4ce4-8df2-1d8552019086 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:13.750609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30453d4-a400-4ce4-8df2-1d8552019086 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:13.752115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30453d4-a400-4ce4-8df2-1d8552019086 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:13.773081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 520/1015] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:13 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:13.847881 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 496/1016] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:05:13 2022] POST /identity/v3/projects => generated 311 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:05:14.045666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636c25f-5325-4474-8f64-b693c55eacfb None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:14.149431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636c25f-5325-4474-8f64-b693c55eacfb None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:14.185201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac59af9-6ba2-4a9e-ad44-54918a56d4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:14.186229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac59af9-6ba2-4a9e-ad44-54918a56d4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:14.186711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac59af9-6ba2-4a9e-ad44-54918a56d4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:14.187186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac59af9-6ba2-4a9e-ad44-54918a56d4e6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:14.188110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac59af9-6ba2-4a9e-ad44-54918a56d4e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:14.189693 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac59af9-6ba2-4a9e-ad44-54918a56d4e6 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:14.192478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-7ac59af9-6ba2-4a9e-ad44-54918a56d4e6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:14.199723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:14.200936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 16:05:14.201964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:14.204381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 521/1017] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:05:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:14.210353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30453d4-a400-4ce4-8df2-1d8552019086 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:14.252405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:14.313373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30453d4-a400-4ce4-8df2-1d8552019086 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:14.348251 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35df2d-79f6-4c63-9682-3e5dac672f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:14.348872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35df2d-79f6-4c63-9682-3e5dac672f98 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:14.349150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35df2d-79f6-4c63-9682-3e5dac672f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:14.349421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35df2d-79f6-4c63-9682-3e5dac672f98 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:14.350049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35df2d-79f6-4c63-9682-3e5dac672f98 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:14.350352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35df2d-79f6-4c63-9682-3e5dac672f98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:14.351738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de35df2d-79f6-4c63-9682-3e5dac672f98 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:14.359035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 497/1018] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:05:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:14.362445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac59af9-6ba2-4a9e-ad44-54918a56d4e6 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:14.467994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac59af9-6ba2-4a9e-ad44-54918a56d4e6 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:14.501213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0501b0a-ea46-45ea-a1a8-e3a9e9cd139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:14.501816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0501b0a-ea46-45ea-a1a8-e3a9e9cd139c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:14.502093 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0501b0a-ea46-45ea-a1a8-e3a9e9cd139c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:14.502368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0501b0a-ea46-45ea-a1a8-e3a9e9cd139c None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:14.502989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0501b0a-ea46-45ea-a1a8-e3a9e9cd139c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:14.503287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0501b0a-ea46-45ea-a1a8-e3a9e9cd139c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:14.504662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0501b0a-ea46-45ea-a1a8-e3a9e9cd139c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:14.509538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 522/1019] 10.222.0.15 () {56 vars in 1112 bytes} [Wed Sep 7 16:05:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:14.515597 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35df2d-79f6-4c63-9682-3e5dac672f98 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:14.619572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35df2d-79f6-4c63-9682-3e5dac672f98 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:14.621920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:14.654390 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e99baf4-ac63-4b31-a383-5a563202abbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:14.655324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99baf4-ac63-4b31-a383-5a563202abbc None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:14.655986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99baf4-ac63-4b31-a383-5a563202abbc None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:14.656431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e99baf4-ac63-4b31-a383-5a563202abbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:14.657245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99baf4-ac63-4b31-a383-5a563202abbc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:14.658768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e99baf4-ac63-4b31-a383-5a563202abbc None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:14.740780 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 498/1020] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:05:14 2022] POST /identity/v3/endpoints => generated 334 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:05:15.853583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848546d2-0acd-4a5a-9b20-6a3ed999de8b None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:15.854124 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848546d2-0acd-4a5a-9b20-6a3ed999de8b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:15.854124 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848546d2-0acd-4a5a-9b20-6a3ed999de8b None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:15.855633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 523/1021] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:15.860621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd02b79-e427-412f-9c20-0891b98b396c None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:15.860994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd02b79-e427-412f-9c20-0891b98b396c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:15.861180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd02b79-e427-412f-9c20-0891b98b396c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:15.879579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7dd02b79-e427-412f-9c20-0891b98b396c None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:15.880801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7dd02b79-e427-412f-9c20-0891b98b396c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:15.959282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 499/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:16.057059 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dec29-7529-4695-8e79-c508ac801f40 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:16.057400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dec29-7529-4695-8e79-c508ac801f40 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:16.057612 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dec29-7529-4695-8e79-c508ac801f40 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:16.059637 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 524/1023] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:16.064805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ac096-f940-4701-a831-71c43fbbbd5a None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:16.065099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ac096-f940-4701-a831-71c43fbbbd5a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:16.065379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ac096-f940-4701-a831-71c43fbbbd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:16.084336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c22ac096-f940-4701-a831-71c43fbbbd5a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:16.085534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c22ac096-f940-4701-a831-71c43fbbbd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:16.154044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 500/1024] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:16.225479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dec29-7529-4695-8e79-c508ac801f40 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:16.288268 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:16.328032 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857dec29-7529-4695-8e79-c508ac801f40 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:16.361028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830a6ac-b6c6-4502-90c2-c4dff40caee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:16.361660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830a6ac-b6c6-4502-90c2-c4dff40caee8 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:16.361938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830a6ac-b6c6-4502-90c2-c4dff40caee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:16.362208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830a6ac-b6c6-4502-90c2-c4dff40caee8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:16.362779 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830a6ac-b6c6-4502-90c2-c4dff40caee8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:16.364173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4830a6ac-b6c6-4502-90c2-c4dff40caee8 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-4830a6ac-b6c6-4502-90c2-c4dff40caee8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:16.373583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:16.374564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 16:05:16.375424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:16.381206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 525/1025] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Sep 7 16:05:16 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:16.381322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ac096-f940-4701-a831-71c43fbbbd5a None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:16.396389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:16.427685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830a6ac-b6c6-4502-90c2-c4dff40caee8 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:16.485137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ac096-f940-4701-a831-71c43fbbbd5a None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:16.521169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a141d38-f9b2-43e0-8305-e725b2ad24fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:16.521783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a141d38-f9b2-43e0-8305-e725b2ad24fa None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:16.522059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a141d38-f9b2-43e0-8305-e725b2ad24fa None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:16.522334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a141d38-f9b2-43e0-8305-e725b2ad24fa None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:16.522965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a141d38-f9b2-43e0-8305-e725b2ad24fa None admin] RBAC: Adding query filter params (name=image) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:16.523265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a141d38-f9b2-43e0-8305-e725b2ad24fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:16.524659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a141d38-f9b2-43e0-8305-e725b2ad24fa None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:16.530256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 501/1026] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:16.535876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a141d38-f9b2-43e0-8305-e725b2ad24fa None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:16.544190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830a6ac-b6c6-4502-90c2-c4dff40caee8 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:16.574160 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:16.582502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b43ad-bd2c-465e-8021-2863c154555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:16.583127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b43ad-bd2c-465e-8021-2863c154555f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:16.583408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b43ad-bd2c-465e-8021-2863c154555f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:16.583676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b43ad-bd2c-465e-8021-2863c154555f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:16.590127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b43ad-bd2c-465e-8021-2863c154555f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:16.590127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b43ad-bd2c-465e-8021-2863c154555f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:16.590127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 526/1027] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:16.594471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b43ad-bd2c-465e-8021-2863c154555f None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:16.621337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:16.644283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a141d38-f9b2-43e0-8305-e725b2ad24fa None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:16.682014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bd7b5-ea39-42fc-9d6b-eeaeaa112faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:16.682555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bd7b5-ea39-42fc-9d6b-eeaeaa112faf None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:16.682835 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bd7b5-ea39-42fc-9d6b-eeaeaa112faf None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:16.683115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bd7b5-ea39-42fc-9d6b-eeaeaa112faf None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:16.683805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bd7b5-ea39-42fc-9d6b-eeaeaa112faf None admin] RBAC: Adding query filter params (type=image) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:16.684115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bd7b5-ea39-42fc-9d6b-eeaeaa112faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:16.685484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2bd7b5-ea39-42fc-9d6b-eeaeaa112faf None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:16.690592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 502/1028] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:16.696441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bd7b5-ea39-42fc-9d6b-eeaeaa112faf None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:16.703402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b43ad-bd2c-465e-8021-2863c154555f None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:16.739131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8813b8-e3e6-4254-96a1-2320258cacaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:16.739739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8813b8-e3e6-4254-96a1-2320258cacaa None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:16.740052 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8813b8-e3e6-4254-96a1-2320258cacaa None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:16.740326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8813b8-e3e6-4254-96a1-2320258cacaa None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:16.740969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8813b8-e3e6-4254-96a1-2320258cacaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:16.742390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8813b8-e3e6-4254-96a1-2320258cacaa None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:16.798853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2bd7b5-ea39-42fc-9d6b-eeaeaa112faf None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:16.832373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb53c44-f0b6-4845-a61b-6d6aaf03dc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:16.832983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb53c44-f0b6-4845-a61b-6d6aaf03dc15 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:16.833264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb53c44-f0b6-4845-a61b-6d6aaf03dc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:16.833538 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb53c44-f0b6-4845-a61b-6d6aaf03dc15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:16.834237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb53c44-f0b6-4845-a61b-6d6aaf03dc15 None admin] RBAC: Adding query filter params (interface=public, service_id=5adce769f74c4602859af5906f8d0d62, region_id=RegionOne) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:16.834552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb53c44-f0b6-4845-a61b-6d6aaf03dc15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:16.836021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb53c44-f0b6-4845-a61b-6d6aaf03dc15 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:16.841670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 503/1029] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 7 16:05:16 2022] GET /identity/v3/endpoints?service_id=5adce769f74c4602859af5906f8d0d62&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:16.847614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb53c44-f0b6-4845-a61b-6d6aaf03dc15 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:16.857945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:16.862430 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 527/1030] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:05:16 2022] POST /identity/v3/users => generated 285 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:05:16.966144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb53c44-f0b6-4845-a61b-6d6aaf03dc15 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:16.972930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cca22-0366-43a7-aa92-bc12f424132b None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:16.973287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cca22-0366-43a7-aa92-bc12f424132b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:16.973494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07cca22-0366-43a7-aa92-bc12f424132b None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:16.974902 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 528/1031] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:05:16.980608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b53777-f209-4bfe-aab8-ace692e62296 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:16.981192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b53777-f209-4bfe-aab8-ace692e62296 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:16.981469 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b53777-f209-4bfe-aab8-ace692e62296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:17.000720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e6b53777-f209-4bfe-aab8-ace692e62296 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:17.001994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e6b53777-f209-4bfe-aab8-ace692e62296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:17.019811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7633e9-e432-4410-bbd6-9e6c97a83c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:17.020548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7633e9-e432-4410-bbd6-9e6c97a83c20 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:17.020831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7633e9-e432-4410-bbd6-9e6c97a83c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:17.021213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7633e9-e432-4410-bbd6-9e6c97a83c20 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:17.021728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7633e9-e432-4410-bbd6-9e6c97a83c20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:17.023105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7633e9-e432-4410-bbd6-9e6c97a83c20 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:17.037546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 504/1032] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 7 16:05:16 2022] GET /identity/v3/services => generated 1957 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:05:17.062424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:17.084959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 529/1033] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:05:17.352185 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7633e9-e432-4410-bbd6-9e6c97a83c20 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:17.427163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:17.460021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7633e9-e432-4410-bbd6-9e6c97a83c20 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:17.494014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d176b3-2b5d-4766-ac25-78fd03b73656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:17.494646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d176b3-2b5d-4766-ac25-78fd03b73656 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:17.494923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d176b3-2b5d-4766-ac25-78fd03b73656 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:17.495191 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d176b3-2b5d-4766-ac25-78fd03b73656 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:17.506076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d176b3-2b5d-4766-ac25-78fd03b73656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-57d176b3-2b5d-4766-ac25-78fd03b73656 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:05:17.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:17.512920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:05:17.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:17.516371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 505/1034] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 7 16:05:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:17.519061 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b53777-f209-4bfe-aab8-ace692e62296 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:17.619271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b53777-f209-4bfe-aab8-ace692e62296 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:17.655081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0bcce-6287-47b0-be99-cc75058e2600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:17.655686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0bcce-6287-47b0-be99-cc75058e2600 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:17.655983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0bcce-6287-47b0-be99-cc75058e2600 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:17.656252 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0bcce-6287-47b0-be99-cc75058e2600 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:17.656908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0bcce-6287-47b0-be99-cc75058e2600 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:17.657206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0bcce-6287-47b0-be99-cc75058e2600 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:17.658556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b0bcce-6287-47b0-be99-cc75058e2600 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:17.671860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 530/1035] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 7 16:05:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:17.673835 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d176b3-2b5d-4766-ac25-78fd03b73656 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:17.776607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d176b3-2b5d-4766-ac25-78fd03b73656 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:17.805605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:17.812632 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabcba95-b803-4a89-8db5-e7e4f0f12efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:17.813293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcba95-b803-4a89-8db5-e7e4f0f12efd None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:17.817644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcba95-b803-4a89-8db5-e7e4f0f12efd None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:17.817644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabcba95-b803-4a89-8db5-e7e4f0f12efd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:17.826720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabcba95-b803-4a89-8db5-e7e4f0f12efd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:17.828253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabcba95-b803-4a89-8db5-e7e4f0f12efd None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-fabcba95-b803-4a89-8db5-e7e4f0f12efd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:17.843702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:17.844720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:05:17.845629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:17.845629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:17.845629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:17.846403 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 506/1036] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:17.850492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0bcce-6287-47b0-be99-cc75058e2600 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:17.897590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:17.954924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0bcce-6287-47b0-be99-cc75058e2600 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:17.987931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262955f-5691-4909-aab2-725f6c417b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:17.988536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262955f-5691-4909-aab2-725f6c417b70 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:17.988807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262955f-5691-4909-aab2-725f6c417b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:17.989076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262955f-5691-4909-aab2-725f6c417b70 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:17.989703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262955f-5691-4909-aab2-725f6c417b70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:17.990004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262955f-5691-4909-aab2-725f6c417b70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:17.991382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262955f-5691-4909-aab2-725f6c417b70 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:18.014949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 531/1037] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:18.016204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabcba95-b803-4a89-8db5-e7e4f0f12efd None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:18.117809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabcba95-b803-4a89-8db5-e7e4f0f12efd None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:18.150841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f7869-3ce0-4141-b53d-6a44953039ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:18.151401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f7869-3ce0-4141-b53d-6a44953039ec None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:18.151678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f7869-3ce0-4141-b53d-6a44953039ec None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:18.152508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f7869-3ce0-4141-b53d-6a44953039ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:18.160020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f7869-3ce0-4141-b53d-6a44953039ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:18.161454 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f7869-3ce0-4141-b53d-6a44953039ec None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-633f7869-3ce0-4141-b53d-6a44953039ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:05:18.172974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:18.173920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:18.174877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:18.174877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:18.174877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:05:18.174877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:05:18.174877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:05:18.174877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:05:18.174877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:05:18.174877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:05:18.174877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:05:18.174877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:18.174877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 507/1038] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 7 16:05:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:18.181831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262955f-5691-4909-aab2-725f6c417b70 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:18.282639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262955f-5691-4909-aab2-725f6c417b70 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:18.284983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:18.316781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e10ce-f29c-4d4f-8484-e24e75d07faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:18.317426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e10ce-f29c-4d4f-8484-e24e75d07faf None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:18.317820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e10ce-f29c-4d4f-8484-e24e75d07faf None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:18.318097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e10ce-f29c-4d4f-8484-e24e75d07faf None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:18.318816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e10ce-f29c-4d4f-8484-e24e75d07faf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:18.319118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e10ce-f29c-4d4f-8484-e24e75d07faf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:18.320566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e10ce-f29c-4d4f-8484-e24e75d07faf None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:18.338527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 532/1039] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 7 16:05:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:18.345019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f7869-3ce0-4141-b53d-6a44953039ec None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:18.389988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:18.447389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f7869-3ce0-4141-b53d-6a44953039ec None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:18.480778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebb6c4-56f7-430c-930f-558fb8306012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:18.481382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebb6c4-56f7-430c-930f-558fb8306012 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:18.481657 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebb6c4-56f7-430c-930f-558fb8306012 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:18.481924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebb6c4-56f7-430c-930f-558fb8306012 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:18.548117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebb6c4-56f7-430c-930f-558fb8306012 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:18.549567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ebb6c4-56f7-430c-930f-558fb8306012 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:18.591560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:18.649847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 508/1040] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:05:18 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 16:05:18.656432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e10ce-f29c-4d4f-8484-e24e75d07faf None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:18.755875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e10ce-f29c-4d4f-8484-e24e75d07faf None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:18.789521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39842c-a193-4858-9207-a1739b54859f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:18.790151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39842c-a193-4858-9207-a1739b54859f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:18.790429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39842c-a193-4858-9207-a1739b54859f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:18.790699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39842c-a193-4858-9207-a1739b54859f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:18.800623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39842c-a193-4858-9207-a1739b54859f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:18.802058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39842c-a193-4858-9207-a1739b54859f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-bc39842c-a193-4858-9207-a1739b54859f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:18.817095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:18.818057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:05:18.819010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:18.819010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:18.819010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:18.819010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 533/1041] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:18.825555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebb6c4-56f7-430c-930f-558fb8306012 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:18.922784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ebb6c4-56f7-430c-930f-558fb8306012 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:18.961479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b42f5f-3180-41f7-a75f-5eaaa027d26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:18.962080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b42f5f-3180-41f7-a75f-5eaaa027d26a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:18.962354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b42f5f-3180-41f7-a75f-5eaaa027d26a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:18.962623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b42f5f-3180-41f7-a75f-5eaaa027d26a None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:18.963248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b42f5f-3180-41f7-a75f-5eaaa027d26a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:18.963545 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b42f5f-3180-41f7-a75f-5eaaa027d26a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:18.964987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b42f5f-3180-41f7-a75f-5eaaa027d26a None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:18.980955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 509/1042] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:18.982321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39842c-a193-4858-9207-a1739b54859f None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:19.011993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:19.101124 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39842c-a193-4858-9207-a1739b54859f None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:19.114632 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c91063-d8e5-4247-b114-8ee69e17e2d2 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:19.115197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c91063-d8e5-4247-b114-8ee69e17e2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:19.115197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c91063-d8e5-4247-b114-8ee69e17e2d2 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:19.116717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 510/1043] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:19.122384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6760f-ee8f-4cdb-a26a-4da41d4cdf95 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:19.122676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6760f-ee8f-4cdb-a26a-4da41d4cdf95 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:19.122960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6760f-ee8f-4cdb-a26a-4da41d4cdf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:19.129646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:19.140714 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06dbf6-5318-4dc1-87b2-af5059c226a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:19.141346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06dbf6-5318-4dc1-87b2-af5059c226a7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:19.141616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06dbf6-5318-4dc1-87b2-af5059c226a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:19.141961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06dbf6-5318-4dc1-87b2-af5059c226a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:19.142450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-91f6760f-ee8f-4cdb-a26a-4da41d4cdf95 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:19.143643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-91f6760f-ee8f-4cdb-a26a-4da41d4cdf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:19.149060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06dbf6-5318-4dc1-87b2-af5059c226a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:19.150480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06dbf6-5318-4dc1-87b2-af5059c226a7 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-1c06dbf6-5318-4dc1-87b2-af5059c226a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:05:19.161877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:19.162847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:19.163805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:05:19.163805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:05:19.163805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:05:19.163805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:05:19.163805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:05:19.163805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:05:19.163805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:19.163805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:19.164240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 534/1044] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:05:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:19.168846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06dbf6-5318-4dc1-87b2-af5059c226a7 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:19.222984 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 511/1045] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:05:19.224338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9690f-c680-4523-b1b7-a3ba27dd245a None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:19.224627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9690f-c680-4523-b1b7-a3ba27dd245a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:19.224899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9690f-c680-4523-b1b7-a3ba27dd245a None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:19.226446 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 512/1046] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:19.233267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19602-8c83-4ab8-86d6-5ba76bad4cb7 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:19.233545 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19602-8c83-4ab8-86d6-5ba76bad4cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:19.233817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e19602-8c83-4ab8-86d6-5ba76bad4cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:19.252646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-a1e19602-8c83-4ab8-86d6-5ba76bad4cb7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:19.253822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a1e19602-8c83-4ab8-86d6-5ba76bad4cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:19.264682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:19.292023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06dbf6-5318-4dc1-87b2-af5059c226a7 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:19.331521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92f4f6-e25a-4907-a3b8-9adfedf6415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:19.331901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 513/1047] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:19.332181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92f4f6-e25a-4907-a3b8-9adfedf6415e None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:19.332518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92f4f6-e25a-4907-a3b8-9adfedf6415e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:19.332799 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92f4f6-e25a-4907-a3b8-9adfedf6415e None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:19.333489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92f4f6-e25a-4907-a3b8-9adfedf6415e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:19.333796 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92f4f6-e25a-4907-a3b8-9adfedf6415e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:19.335283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92f4f6-e25a-4907-a3b8-9adfedf6415e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:19.347702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 535/1048] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:05:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:19.356196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19602-8c83-4ab8-86d6-5ba76bad4cb7 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:19.430830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:19.476986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e19602-8c83-4ab8-86d6-5ba76bad4cb7 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:19.514158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92f4f6-e25a-4907-a3b8-9adfedf6415e None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:19.515756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dad6c-f50d-48ed-b846-16874cdf5832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:19.516395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dad6c-f50d-48ed-b846-16874cdf5832 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:19.516676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dad6c-f50d-48ed-b846-16874cdf5832 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:19.516950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dad6c-f50d-48ed-b846-16874cdf5832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:19.517676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dad6c-f50d-48ed-b846-16874cdf5832 None admin] RBAC: Adding query filter params (role.id=2c8c5f15b3624270a26334ed3ebdf44a, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=45061db68b1e40f59935cc9a46d2ff10) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:19.517983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dad6c-f50d-48ed-b846-16874cdf5832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:19.519407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24dad6c-f50d-48ed-b846-16874cdf5832 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:19.532137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 514/1049] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:05:19 2022] GET /identity/v3/role_assignments?user.id=45061db68b1e40f59935cc9a46d2ff10&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=2c8c5f15b3624270a26334ed3ebdf44a => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:19.614395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dad6c-f50d-48ed-b846-16874cdf5832 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:19.635365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92f4f6-e25a-4907-a3b8-9adfedf6415e None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:19.680862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc339500-ca8c-4985-8b1b-eccd28f554b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:19.681526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc339500-ca8c-4985-8b1b-eccd28f554b1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:19.682099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc339500-ca8c-4985-8b1b-eccd28f554b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:19.682382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc339500-ca8c-4985-8b1b-eccd28f554b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:19.685421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:19.698492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc339500-ca8c-4985-8b1b-eccd28f554b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-bc339500-ca8c-4985-8b1b-eccd28f554b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:05:19.705946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:19.706903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:19.707757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:19.709040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 536/1050] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:05:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:19.714985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc339500-ca8c-4985-8b1b-eccd28f554b1 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:19.746358 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dad6c-f50d-48ed-b846-16874cdf5832 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:19.782351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:19.790925 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2061e0b-e5b2-4083-9c01-f98a4f7d9b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:19.791559 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2061e0b-e5b2-4083-9c01-f98a4f7d9b26 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:19.791866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2061e0b-e5b2-4083-9c01-f98a4f7d9b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:19.792145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2061e0b-e5b2-4083-9c01-f98a4f7d9b26 None admin] PATH_INFO: `/v3/services/image` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:19.793359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2061e0b-e5b2-4083-9c01-f98a4f7d9b26 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:19.796802 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2061e0b-e5b2-4083-9c01-f98a4f7d9b26 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-e2061e0b-e5b2-4083-9c01-f98a4f7d9b26 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:19.806361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:19.807437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:19.809245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 515/1051] 10.222.0.15 () {56 vars in 1105 bytes} [Wed Sep 7 16:05:19 2022] GET /identity/v3/services/image => generated 86 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:19.813988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2061e0b-e5b2-4083-9c01-f98a4f7d9b26 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:19.851160 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc339500-ca8c-4985-8b1b-eccd28f554b1 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:19.892815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a351a-44e9-4b52-a7c7-6a4a88d744d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:19.893432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a351a-44e9-4b52-a7c7-6a4a88d744d4 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:19.893714 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a351a-44e9-4b52-a7c7-6a4a88d744d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:19.893991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a351a-44e9-4b52-a7c7-6a4a88d744d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:19.894655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a351a-44e9-4b52-a7c7-6a4a88d744d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:19.894961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a351a-44e9-4b52-a7c7-6a4a88d744d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:19.896363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a351a-44e9-4b52-a7c7-6a4a88d744d4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:19.915018 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 537/1052] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:05:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:19.916165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a351a-44e9-4b52-a7c7-6a4a88d744d4 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:19.938619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2061e0b-e5b2-4083-9c01-f98a4f7d9b26 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:19.980391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376535b-d597-452e-b22c-4c60ad38a711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:19.981007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376535b-d597-452e-b22c-4c60ad38a711 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:19.981289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376535b-d597-452e-b22c-4c60ad38a711 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:19.981569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376535b-d597-452e-b22c-4c60ad38a711 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:19.982199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376535b-d597-452e-b22c-4c60ad38a711 None admin] RBAC: Adding query filter params (name=image) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:19.982503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376535b-d597-452e-b22c-4c60ad38a711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:19.983900 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376535b-d597-452e-b22c-4c60ad38a711 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:19.986034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:19.989559 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 516/1053] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:19.994810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376535b-d597-452e-b22c-4c60ad38a711 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:20.042134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a351a-44e9-4b52-a7c7-6a4a88d744d4 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:20.053741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:20.078152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb946c-409a-44c6-9611-ed462b6b418a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:20.078779 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb946c-409a-44c6-9611-ed462b6b418a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:20.079059 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb946c-409a-44c6-9611-ed462b6b418a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:20.079336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb946c-409a-44c6-9611-ed462b6b418a None admin] PATH_INFO: `/v3/users/b076537b32ee409ca57da121141e3efc` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:20.094199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb946c-409a-44c6-9611-ed462b6b418a None admin] RBAC: Authorizing `identity:get_user(user_id=b076537b32ee409ca57da121141e3efc)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:20.095644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb946c-409a-44c6-9611-ed462b6b418a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:20.097284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 538/1054] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:19 2022] GET /identity/v3/users/b076537b32ee409ca57da121141e3efc => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:20.103275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb946c-409a-44c6-9611-ed462b6b418a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:20.106432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376535b-d597-452e-b22c-4c60ad38a711 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:20.150510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb790a5-619f-4d16-85a9-8174eb89bc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:20.151157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb790a5-619f-4d16-85a9-8174eb89bc69 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:20.151157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb790a5-619f-4d16-85a9-8174eb89bc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:20.151395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb790a5-619f-4d16-85a9-8174eb89bc69 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:20.152089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb790a5-619f-4d16-85a9-8174eb89bc69 None admin] RBAC: Adding query filter params (type=image) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:20.152393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb790a5-619f-4d16-85a9-8174eb89bc69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:20.153816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb790a5-619f-4d16-85a9-8174eb89bc69 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:20.163358 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 517/1055] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:20.164951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb790a5-619f-4d16-85a9-8174eb89bc69 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:20.209241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb946c-409a-44c6-9611-ed462b6b418a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:20.225711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:20.247557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b3ecb-a767-4412-940f-4641c280cdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:20.248213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3ecb-a767-4412-940f-4641c280cdc1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:20.248498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3ecb-a767-4412-940f-4641c280cdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:20.248773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223b3ecb-a767-4412-940f-4641c280cdc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:20.272674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb790a5-619f-4d16-85a9-8174eb89bc69 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:20.291221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:20.309918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af766c5-3e21-461d-a536-fc8d18318127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:20.310456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af766c5-3e21-461d-a536-fc8d18318127 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:20.310732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af766c5-3e21-461d-a536-fc8d18318127 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:20.310997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af766c5-3e21-461d-a536-fc8d18318127 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:20.311655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af766c5-3e21-461d-a536-fc8d18318127 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:20.313147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af766c5-3e21-461d-a536-fc8d18318127 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:20.324075 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b3ecb-a767-4412-940f-4641c280cdc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:20.325580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223b3ecb-a767-4412-940f-4641c280cdc1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:20.423304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 539/1056] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:05:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:05:20.429038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b3ecb-a767-4412-940f-4641c280cdc1 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:20.438175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 518/1057] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:05:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:05:20.449164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:20.541333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223b3ecb-a767-4412-940f-4641c280cdc1 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:20.588208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe47935-31ed-478a-8584-178bedade17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:20.588776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe47935-31ed-478a-8584-178bedade17e None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:20.589060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe47935-31ed-478a-8584-178bedade17e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:20.589901 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe47935-31ed-478a-8584-178bedade17e None admin] PATH_INFO: `/v3/projects/e125dbb56cc445bbb4360578df050214` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:20.591056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe47935-31ed-478a-8584-178bedade17e None admin] RBAC: Authorizing `identity:get_project(project_id=e125dbb56cc445bbb4360578df050214)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:20.592527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe47935-31ed-478a-8584-178bedade17e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:20.594374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 540/1058] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:05:20 2022] GET /identity/v3/projects/e125dbb56cc445bbb4360578df050214 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:20.600437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af766c5-3e21-461d-a536-fc8d18318127 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:20.709180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af766c5-3e21-461d-a536-fc8d18318127 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:20.742572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb31f0-cb6d-4d96-9ade-5eb180ab2d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:20.743176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb31f0-cb6d-4d96-9ade-5eb180ab2d86 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:20.743447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb31f0-cb6d-4d96-9ade-5eb180ab2d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:20.743710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb31f0-cb6d-4d96-9ade-5eb180ab2d86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:20.744448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb31f0-cb6d-4d96-9ade-5eb180ab2d86 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=e125dbb56cc445bbb4360578df050214, user.id=b076537b32ee409ca57da121141e3efc) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:20.744759 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb31f0-cb6d-4d96-9ade-5eb180ab2d86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:20.746721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb31f0-cb6d-4d96-9ade-5eb180ab2d86 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:20.752351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 519/1059] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:05:20 2022] GET /identity/v3/role_assignments?user.id=b076537b32ee409ca57da121141e3efc&scope.project.id=e125dbb56cc445bbb4360578df050214&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:21.706488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75478b-ea15-41b6-9209-7607ee7eb2ea None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:21.707005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75478b-ea15-41b6-9209-7607ee7eb2ea None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:21.707083 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75478b-ea15-41b6-9209-7607ee7eb2ea None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:21.708429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 541/1060] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:21.713529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96ab16-728c-4d4f-8133-fd2a471234b3 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:21.713909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96ab16-728c-4d4f-8133-fd2a471234b3 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:21.714076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96ab16-728c-4d4f-8133-fd2a471234b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:21.732091 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ac96ab16-728c-4d4f-8133-fd2a471234b3 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:21.733294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ac96ab16-728c-4d4f-8133-fd2a471234b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:21.763853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:21.801068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 520/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:22.069780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75478b-ea15-41b6-9209-7607ee7eb2ea None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:22.156179 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:22.182317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75478b-ea15-41b6-9209-7607ee7eb2ea None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:22.219010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f6ffb-2935-4647-a241-f407aceb8aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:22.219961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f6ffb-2935-4647-a241-f407aceb8aab None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:22.220425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f6ffb-2935-4647-a241-f407aceb8aab None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:22.220880 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f6ffb-2935-4647-a241-f407aceb8aab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:22.232007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f6ffb-2935-4647-a241-f407aceb8aab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-ed3f6ffb-2935-4647-a241-f407aceb8aab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:05:22.237055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:22.237985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:05:22.238839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:22.240780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 542/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 7 16:05:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:22.246971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96ab16-728c-4d4f-8133-fd2a471234b3 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:22.348419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96ab16-728c-4d4f-8133-fd2a471234b3 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:22.382536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea820ee9-7ead-4429-a42c-299630edb08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:22.383147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea820ee9-7ead-4429-a42c-299630edb08b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:22.383426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea820ee9-7ead-4429-a42c-299630edb08b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:22.383697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea820ee9-7ead-4429-a42c-299630edb08b None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:22.384397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea820ee9-7ead-4429-a42c-299630edb08b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:22.384704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea820ee9-7ead-4429-a42c-299630edb08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:22.386091 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea820ee9-7ead-4429-a42c-299630edb08b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:22.398748 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 521/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 7 16:05:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:22.400540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f6ffb-2935-4647-a241-f407aceb8aab None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:22.505985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f6ffb-2935-4647-a241-f407aceb8aab None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:22.549225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f0ca3-f2a9-49f5-b22d-d3f4638ae073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:22.549872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f0ca3-f2a9-49f5-b22d-d3f4638ae073 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:22.550151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f0ca3-f2a9-49f5-b22d-d3f4638ae073 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:22.550423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f0ca3-f2a9-49f5-b22d-d3f4638ae073 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:22.557598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:22.561944 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f0ca3-f2a9-49f5-b22d-d3f4638ae073 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:22.563391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f0ca3-f2a9-49f5-b22d-d3f4638ae073 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-db9f0ca3-f2a9-49f5-b22d-d3f4638ae073 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:22.578370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:22.579312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:05:22.580319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:22.580319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:22.580319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:22.584082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 543/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:22.585300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea820ee9-7ead-4429-a42c-299630edb08b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:22.618975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:22.635099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c019-cdb8-49a4-a094-46d5e3dd9dd0 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:22.635374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c019-cdb8-49a4-a094-46d5e3dd9dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:22.635651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105c019-cdb8-49a4-a094-46d5e3dd9dd0 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:22.637680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 544/1065] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:05:22.642537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f63ae9-f6aa-4c31-8472-db9b3378eac6 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:22.642821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f63ae9-f6aa-4c31-8472-db9b3378eac6 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:22.643096 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f63ae9-f6aa-4c31-8472-db9b3378eac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:22.661385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-07f63ae9-f6aa-4c31-8472-db9b3378eac6 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:22.662569 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-07f63ae9-f6aa-4c31-8472-db9b3378eac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:22.692245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea820ee9-7ead-4429-a42c-299630edb08b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:22.728582 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1e1756-3efe-4850-b74f-81691b99cb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:22.729231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e1756-3efe-4850-b74f-81691b99cb6d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:22.729596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e1756-3efe-4850-b74f-81691b99cb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:22.729860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1e1756-3efe-4850-b74f-81691b99cb6d None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:22.730494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e1756-3efe-4850-b74f-81691b99cb6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:22.730742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e1756-3efe-4850-b74f-81691b99cb6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:22.732120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1e1756-3efe-4850-b74f-81691b99cb6d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:22.735809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 545/1066] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:05:22.743804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 522/1067] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:22.749590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f63ae9-f6aa-4c31-8472-db9b3378eac6 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:22.852922 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f63ae9-f6aa-4c31-8472-db9b3378eac6 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:22.872794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd57782-d3c1-483d-91db-bd4f03a54ea4 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:22.873007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd57782-d3c1-483d-91db-bd4f03a54ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:22.873275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd57782-d3c1-483d-91db-bd4f03a54ea4 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:22.873564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:22.874730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 523/1068] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:05:22.879335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc4a10-05fd-43ab-9647-e9e1396a326e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:22.879612 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc4a10-05fd-43ab-9647-e9e1396a326e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:22.879931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc4a10-05fd-43ab-9647-e9e1396a326e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:22.889668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afba742-cf42-4062-bf03-225a64b4c2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:22.890301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afba742-cf42-4062-bf03-225a64b4c2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:22.890582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afba742-cf42-4062-bf03-225a64b4c2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:22.890857 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afba742-cf42-4062-bf03-225a64b4c2d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:22.898391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afba742-cf42-4062-bf03-225a64b4c2d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:22.898534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-41fc4a10-05fd-43ab-9647-e9e1396a326e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:22.899558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-41fc4a10-05fd-43ab-9647-e9e1396a326e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:22.899879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afba742-cf42-4062-bf03-225a64b4c2d4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-0afba742-cf42-4062-bf03-225a64b4c2d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:05:22.912641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:22.913628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:22.914603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:22.914603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:22.914603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:05:22.914603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:05:22.914603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:05:22.914603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:05:22.914603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:05:22.914603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:05:22.914603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:05:22.914603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:22.914603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 546/1069] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 7 16:05:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:22.919480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afba742-cf42-4062-bf03-225a64b4c2d4 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:22.975632 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 524/1070] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:05:23.021891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc4a10-05fd-43ab-9647-e9e1396a326e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:23.029175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afba742-cf42-4062-bf03-225a64b4c2d4 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:23.050581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:23.069106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e53ca-8fe0-4bc4-89ed-453b275c07d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:23.069722 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e53ca-8fe0-4bc4-89ed-453b275c07d2 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:23.070004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e53ca-8fe0-4bc4-89ed-453b275c07d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:23.070279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e53ca-8fe0-4bc4-89ed-453b275c07d2 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:23.071079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e53ca-8fe0-4bc4-89ed-453b275c07d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:23.071391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e53ca-8fe0-4bc4-89ed-453b275c07d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:23.072861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e53ca-8fe0-4bc4-89ed-453b275c07d2 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:23.091087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 547/1071] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 7 16:05:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:23.096931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e53ca-8fe0-4bc4-89ed-453b275c07d2 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:23.144315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc4a10-05fd-43ab-9647-e9e1396a326e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:23.144533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:23.181477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b949c-7e7a-41e1-ac44-5e7e35f00d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:23.182091 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b949c-7e7a-41e1-ac44-5e7e35f00d4e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:23.182376 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b949c-7e7a-41e1-ac44-5e7e35f00d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:23.182652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b949c-7e7a-41e1-ac44-5e7e35f00d4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:23.213041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e53ca-8fe0-4bc4-89ed-453b275c07d2 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:23.249237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162f96b-8566-45a0-bf54-8e9e32fa5945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:23.249869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162f96b-8566-45a0-bf54-8e9e32fa5945 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:23.250149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162f96b-8566-45a0-bf54-8e9e32fa5945 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:23.250425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162f96b-8566-45a0-bf54-8e9e32fa5945 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:23.258770 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b949c-7e7a-41e1-ac44-5e7e35f00d4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:23.260244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b949c-7e7a-41e1-ac44-5e7e35f00d4e None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:23.260592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162f96b-8566-45a0-bf54-8e9e32fa5945 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:23.262612 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f162f96b-8566-45a0-bf54-8e9e32fa5945 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-f162f96b-8566-45a0-bf54-8e9e32fa5945 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:23.278024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:23.279023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:05:23.279952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:23.279952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:23.279952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:23.280747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:23.283902 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 548/1072] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:23.284783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162f96b-8566-45a0-bf54-8e9e32fa5945 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:23.357736 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 525/1073] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:05:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 16:05:23.358556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b949c-7e7a-41e1-ac44-5e7e35f00d4e None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:23.376397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:23.385510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162f96b-8566-45a0-bf54-8e9e32fa5945 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:23.419466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945ad55-4927-4c29-a1c0-274e2e32f587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:23.420168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945ad55-4927-4c29-a1c0-274e2e32f587 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:23.420448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945ad55-4927-4c29-a1c0-274e2e32f587 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:23.420723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945ad55-4927-4c29-a1c0-274e2e32f587 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:23.430793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945ad55-4927-4c29-a1c0-274e2e32f587 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-4945ad55-4927-4c29-a1c0-274e2e32f587 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:05:23.436169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:23.437175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:23.438060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 549/1074] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:05:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:23.438778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945ad55-4927-4c29-a1c0-274e2e32f587 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:23.461144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b949c-7e7a-41e1-ac44-5e7e35f00d4e None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:23.485231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:23.500324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367741ba-37e9-40ec-88ef-c9983db31376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:23.500927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367741ba-37e9-40ec-88ef-c9983db31376 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:23.501207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367741ba-37e9-40ec-88ef-c9983db31376 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:23.501476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367741ba-37e9-40ec-88ef-c9983db31376 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:23.502113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367741ba-37e9-40ec-88ef-c9983db31376 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:23.502411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367741ba-37e9-40ec-88ef-c9983db31376 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:23.503755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367741ba-37e9-40ec-88ef-c9983db31376 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:23.515261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 526/1075] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:23.516071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367741ba-37e9-40ec-88ef-c9983db31376 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:23.539508 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945ad55-4927-4c29-a1c0-274e2e32f587 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:23.573169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63f84-51dd-4e5c-96e3-b853341cce2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:23.573820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63f84-51dd-4e5c-96e3-b853341cce2e None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:23.574094 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63f84-51dd-4e5c-96e3-b853341cce2e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:23.574369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e63f84-51dd-4e5c-96e3-b853341cce2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:23.584467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e63f84-51dd-4e5c-96e3-b853341cce2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:23.585889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e63f84-51dd-4e5c-96e3-b853341cce2e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-12e63f84-51dd-4e5c-96e3-b853341cce2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:23.599691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:23.600720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:05:23.601632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:23.601632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:23.601632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:23.601632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 550/1076] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:23.601904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63f84-51dd-4e5c-96e3-b853341cce2e None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:23.605269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:23.621315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367741ba-37e9-40ec-88ef-c9983db31376 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:23.654816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac327a13-aa3c-4972-8bbb-aa35b098751a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:23.655417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac327a13-aa3c-4972-8bbb-aa35b098751a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:23.655695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac327a13-aa3c-4972-8bbb-aa35b098751a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:23.656001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac327a13-aa3c-4972-8bbb-aa35b098751a None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:23.656655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac327a13-aa3c-4972-8bbb-aa35b098751a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:23.656961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac327a13-aa3c-4972-8bbb-aa35b098751a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:23.658312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac327a13-aa3c-4972-8bbb-aa35b098751a None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:23.666139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 527/1077] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:05:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:23.666899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac327a13-aa3c-4972-8bbb-aa35b098751a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:23.702005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e63f84-51dd-4e5c-96e3-b853341cce2e None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:23.709125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:23.735209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c07617-d823-42c7-99e2-65b9af130c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:23.735893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c07617-d823-42c7-99e2-65b9af130c56 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:23.736456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c07617-d823-42c7-99e2-65b9af130c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:23.736456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c07617-d823-42c7-99e2-65b9af130c56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:23.743249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c07617-d823-42c7-99e2-65b9af130c56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:23.744719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c07617-d823-42c7-99e2-65b9af130c56 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-62c07617-d823-42c7-99e2-65b9af130c56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:05:23.755785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:23.756701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:23.757638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:05:23.757638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:05:23.757638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:05:23.757638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:05:23.757638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:05:23.757638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:05:23.757638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:23.757638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:23.757638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 551/1078] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:23.757638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c07617-d823-42c7-99e2-65b9af130c56 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:23.771245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac327a13-aa3c-4972-8bbb-aa35b098751a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:23.805505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aaaa34-af36-4712-a03a-46258f50f874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:23.806103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aaaa34-af36-4712-a03a-46258f50f874 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:23.806377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aaaa34-af36-4712-a03a-46258f50f874 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:23.806652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aaaa34-af36-4712-a03a-46258f50f874 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:23.807274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aaaa34-af36-4712-a03a-46258f50f874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:23.807572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aaaa34-af36-4712-a03a-46258f50f874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:23.808975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aaaa34-af36-4712-a03a-46258f50f874 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:23.818350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:23.821246 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 528/1079] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:23.822035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aaaa34-af36-4712-a03a-46258f50f874 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:23.864532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c07617-d823-42c7-99e2-65b9af130c56 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:23.904080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d948bf-0ecd-4afd-9482-94b37c7e98d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:23.904981 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d948bf-0ecd-4afd-9482-94b37c7e98d3 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:23.905442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d948bf-0ecd-4afd-9482-94b37c7e98d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:23.905857 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d948bf-0ecd-4afd-9482-94b37c7e98d3 None admin] PATH_INFO: `/v3/users/b076537b32ee409ca57da121141e3efc` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:23.907221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d948bf-0ecd-4afd-9482-94b37c7e98d3 None admin] RBAC: Authorizing `identity:get_user(user_id=b076537b32ee409ca57da121141e3efc)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:23.908704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d948bf-0ecd-4afd-9482-94b37c7e98d3 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:23.910502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 552/1080] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:23 2022] GET /identity/v3/users/b076537b32ee409ca57da121141e3efc => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:23.916329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d948bf-0ecd-4afd-9482-94b37c7e98d3 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:23.945501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aaaa34-af36-4712-a03a-46258f50f874 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:23.947545 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:23.991035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9cd5f-e8f1-4aaf-b187-1b14497dd468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:23.991572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9cd5f-e8f1-4aaf-b187-1b14497dd468 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:23.991930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9cd5f-e8f1-4aaf-b187-1b14497dd468 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:23.992252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a9cd5f-e8f1-4aaf-b187-1b14497dd468 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:23.993024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9cd5f-e8f1-4aaf-b187-1b14497dd468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:23.993331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9cd5f-e8f1-4aaf-b187-1b14497dd468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:23.995240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a9cd5f-e8f1-4aaf-b187-1b14497dd468 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:24.009114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 529/1081] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:24.015833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9cd5f-e8f1-4aaf-b187-1b14497dd468 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:24.040922 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d948bf-0ecd-4afd-9482-94b37c7e98d3 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:24.076233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:24.083941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbfc69-86ee-4bd2-bfda-883c1e6df2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:24.088148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbfc69-86ee-4bd2-bfda-883c1e6df2b4 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:24.088148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbfc69-86ee-4bd2-bfda-883c1e6df2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:24.088148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbfc69-86ee-4bd2-bfda-883c1e6df2b4 None admin] PATH_INFO: `/v3/projects/e125dbb56cc445bbb4360578df050214` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:24.088148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbfc69-86ee-4bd2-bfda-883c1e6df2b4 None admin] RBAC: Authorizing `identity:get_project(project_id=e125dbb56cc445bbb4360578df050214)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:24.088148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbfc69-86ee-4bd2-bfda-883c1e6df2b4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:24.091262 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 553/1082] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:05:23 2022] GET /identity/v3/projects/e125dbb56cc445bbb4360578df050214 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:24.095844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbfc69-86ee-4bd2-bfda-883c1e6df2b4 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:24.140266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a9cd5f-e8f1-4aaf-b187-1b14497dd468 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:24.173562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014a0e7-74da-41f0-bfcd-496256907ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:24.174220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014a0e7-74da-41f0-bfcd-496256907ab3 None admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:24.174502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014a0e7-74da-41f0-bfcd-496256907ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:24.174775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014a0e7-74da-41f0-bfcd-496256907ab3 None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/45061db68b1e40f59935cc9a46d2ff10/roles/2c8c5f15b3624270a26334ed3ebdf44a` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:24.176935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014a0e7-74da-41f0-bfcd-496256907ab3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=45061db68b1e40f59935cc9a46d2ff10, role_id=2c8c5f15b3624270a26334ed3ebdf44a)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:24.178366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014a0e7-74da-41f0-bfcd-496256907ab3 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:24.201159 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbfc69-86ee-4bd2-bfda-883c1e6df2b4 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:24.233864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7034c0-54dd-407d-92a6-ef9bb7c38945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:24.234503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7034c0-54dd-407d-92a6-ef9bb7c38945 None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:24.234778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7034c0-54dd-407d-92a6-ef9bb7c38945 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:24.235053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7034c0-54dd-407d-92a6-ef9bb7c38945 None admin] PATH_INFO: `/v3/projects/e125dbb56cc445bbb4360578df050214/users/b076537b32ee409ca57da121141e3efc/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:24.236938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7034c0-54dd-407d-92a6-ef9bb7c38945 None admin] RBAC: Authorizing `identity:create_grant(project_id=e125dbb56cc445bbb4360578df050214, user_id=b076537b32ee409ca57da121141e3efc, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:24.238382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7034c0-54dd-407d-92a6-ef9bb7c38945 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:24.244249 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 530/1083] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:05:24 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/45061db68b1e40f59935cc9a46d2ff10/roles/2c8c5f15b3624270a26334ed3ebdf44a => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:05:24.331451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 554/1084] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:05:24 2022] PUT /identity/v3/projects/e125dbb56cc445bbb4360578df050214/users/b076537b32ee409ca57da121141e3efc/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:05:26.369449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867df276-13be-420a-8eeb-17b3a1f8faf5 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:26.370693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867df276-13be-420a-8eeb-17b3a1f8faf5 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:26.370693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867df276-13be-420a-8eeb-17b3a1f8faf5 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:26.371929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 531/1085] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:26.377367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e7cbd-26a8-4b2a-9921-2ce5895fc353 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:26.377800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e7cbd-26a8-4b2a-9921-2ce5895fc353 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:26.377903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e7cbd-26a8-4b2a-9921-2ce5895fc353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:26.395734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-427e7cbd-26a8-4b2a-9921-2ce5895fc353 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:26.396933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-427e7cbd-26a8-4b2a-9921-2ce5895fc353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:26.449195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:26.455067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e9367-6e14-437d-a9ab-79781f0d1c6d None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:26.455481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e9367-6e14-437d-a9ab-79781f0d1c6d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:26.455680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e9367-6e14-437d-a9ab-79781f0d1c6d None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:26.457035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 532/1086] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:05:26.462051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef7f45-03b9-4bb9-8d64-b750c58c8975 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:26.462335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef7f45-03b9-4bb9-8d64-b750c58c8975 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:26.462612 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef7f45-03b9-4bb9-8d64-b750c58c8975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:26.462780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 555/1087] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:26.480679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-f6ef7f45-03b9-4bb9-8d64-b750c58c8975 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:26.481851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f6ef7f45-03b9-4bb9-8d64-b750c58c8975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:26.498588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:26.551534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 533/1088] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:05:26.730533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e7cbd-26a8-4b2a-9921-2ce5895fc353 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:26.818355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef7f45-03b9-4bb9-8d64-b750c58c8975 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:26.828237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e7cbd-26a8-4b2a-9921-2ce5895fc353 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:26.862010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823e2c-daf7-4f22-947e-67bb0dc5dd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:26.862878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823e2c-daf7-4f22-947e-67bb0dc5dd1c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:26.863152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823e2c-daf7-4f22-947e-67bb0dc5dd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:26.863420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823e2c-daf7-4f22-947e-67bb0dc5dd1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:26.873365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49823e2c-daf7-4f22-947e-67bb0dc5dd1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-49823e2c-daf7-4f22-947e-67bb0dc5dd1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:05:26.878162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:26.879413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:26.880542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 556/1089] 10.222.0.15 () {56 vars in 1120 bytes} [Wed Sep 7 16:05:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:26.885119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823e2c-daf7-4f22-947e-67bb0dc5dd1c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:26.932207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef7f45-03b9-4bb9-8d64-b750c58c8975 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:26.936698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:26.965557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891864e8-7802-427d-a8ff-f70552759818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:26.966179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891864e8-7802-427d-a8ff-f70552759818 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:26.966454 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891864e8-7802-427d-a8ff-f70552759818 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:26.966721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891864e8-7802-427d-a8ff-f70552759818 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:26.977289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891864e8-7802-427d-a8ff-f70552759818 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-891864e8-7802-427d-a8ff-f70552759818 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:05:26.984079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:26.985009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:26.985854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 534/1090] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:05:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:26.989917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891864e8-7802-427d-a8ff-f70552759818 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:26.996011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823e2c-daf7-4f22-947e-67bb0dc5dd1c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:27.005193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:27.032978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3354f26-6b54-4ac9-af2f-9a4cac0f6c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:27.033583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3354f26-6b54-4ac9-af2f-9a4cac0f6c56 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:27.033864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3354f26-6b54-4ac9-af2f-9a4cac0f6c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:27.034132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3354f26-6b54-4ac9-af2f-9a4cac0f6c56 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:27.034781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3354f26-6b54-4ac9-af2f-9a4cac0f6c56 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:27.035080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3354f26-6b54-4ac9-af2f-9a4cac0f6c56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:27.036497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3354f26-6b54-4ac9-af2f-9a4cac0f6c56 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:27.044674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 557/1091] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 7 16:05:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:27.050438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3354f26-6b54-4ac9-af2f-9a4cac0f6c56 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:27.094514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891864e8-7802-427d-a8ff-f70552759818 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:27.128316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89781460-d501-46b0-9021-f1619fb7d684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:27.129027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781460-d501-46b0-9021-f1619fb7d684 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:27.129299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781460-d501-46b0-9021-f1619fb7d684 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:27.129571 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89781460-d501-46b0-9021-f1619fb7d684 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:27.130227 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89781460-d501-46b0-9021-f1619fb7d684 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:27.130526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89781460-d501-46b0-9021-f1619fb7d684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:27.131908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89781460-d501-46b0-9021-f1619fb7d684 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:27.140728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 535/1092] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:05:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:27.146890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89781460-d501-46b0-9021-f1619fb7d684 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:27.152001 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3354f26-6b54-4ac9-af2f-9a4cac0f6c56 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:27.162141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:27.184912 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cbf55-3fb6-4d3b-b6fd-21c512121aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:27.185532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cbf55-3fb6-4d3b-b6fd-21c512121aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:27.185810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cbf55-3fb6-4d3b-b6fd-21c512121aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:27.186081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cbf55-3fb6-4d3b-b6fd-21c512121aa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:27.196220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0cbf55-3fb6-4d3b-b6fd-21c512121aa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:27.197666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0cbf55-3fb6-4d3b-b6fd-21c512121aa1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-aa0cbf55-3fb6-4d3b-b6fd-21c512121aa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:27.213205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:27.215009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:05:27.220528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:27.220528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:27.220528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:27.220528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 558/1093] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:27.220891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cbf55-3fb6-4d3b-b6fd-21c512121aa1 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:27.227133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:27.251558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89781460-d501-46b0-9021-f1619fb7d684 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:27.286544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d39282-c1bc-40ff-9ee2-b035231d2d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:27.287164 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d39282-c1bc-40ff-9ee2-b035231d2d9c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:27.287439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d39282-c1bc-40ff-9ee2-b035231d2d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:27.287711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d39282-c1bc-40ff-9ee2-b035231d2d9c None admin] PATH_INFO: `/v3/users/b076537b32ee409ca57da121141e3efc` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:27.288867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d39282-c1bc-40ff-9ee2-b035231d2d9c None admin] RBAC: Authorizing `identity:get_user(user_id=b076537b32ee409ca57da121141e3efc)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:27.290281 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d39282-c1bc-40ff-9ee2-b035231d2d9c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:27.296709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 536/1094] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:27 2022] GET /identity/v3/users/b076537b32ee409ca57da121141e3efc => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:27.297685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d39282-c1bc-40ff-9ee2-b035231d2d9c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:27.325086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cbf55-3fb6-4d3b-b6fd-21c512121aa1 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:27.358957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c53db-d711-4ea3-a2b2-1d55965b32ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:27.359562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c53db-d711-4ea3-a2b2-1d55965b32ea None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:27.359892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c53db-d711-4ea3-a2b2-1d55965b32ea None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:27.360157 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c53db-d711-4ea3-a2b2-1d55965b32ea None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:27.360793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c53db-d711-4ea3-a2b2-1d55965b32ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:27.361097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c53db-d711-4ea3-a2b2-1d55965b32ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:27.362456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c53db-d711-4ea3-a2b2-1d55965b32ea None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:27.373981 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 559/1095] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:27.379538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c53db-d711-4ea3-a2b2-1d55965b32ea None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:27.397841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d39282-c1bc-40ff-9ee2-b035231d2d9c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:27.403485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:27.430936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c659cf1-8cdf-44d5-b0b7-5929e97c5792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:27.431539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c659cf1-8cdf-44d5-b0b7-5929e97c5792 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:27.431843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c659cf1-8cdf-44d5-b0b7-5929e97c5792 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:27.432115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c659cf1-8cdf-44d5-b0b7-5929e97c5792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:27.444267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:27.476615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c53db-d711-4ea3-a2b2-1d55965b32ea None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:27.499978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c659cf1-8cdf-44d5-b0b7-5929e97c5792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:27.501412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c659cf1-8cdf-44d5-b0b7-5929e97c5792 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:27.512020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2fc04-a33f-43dd-b33b-b4e1f60b7497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:27.512576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2fc04-a33f-43dd-b33b-b4e1f60b7497 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:27.512853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2fc04-a33f-43dd-b33b-b4e1f60b7497 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:27.513145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2fc04-a33f-43dd-b33b-b4e1f60b7497 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:27.520048 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2fc04-a33f-43dd-b33b-b4e1f60b7497 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:27.521451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2fc04-a33f-43dd-b33b-b4e1f60b7497 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-8fe2fc04-a33f-43dd-b33b-b4e1f60b7497 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:05:27.532674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:27.533852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:27.534876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:27.534876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:27.534876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:05:27.534876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:05:27.534876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:05:27.534876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:05:27.534876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:05:27.534876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:05:27.534876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 16:05:27.534876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:27.534876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 560/1096] 10.222.0.15 () {56 vars in 1093 bytes} [Wed Sep 7 16:05:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:05:27.539367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2fc04-a33f-43dd-b33b-b4e1f60b7497 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:27.606236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 537/1097] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:05:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 16:05:27.612641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c659cf1-8cdf-44d5-b0b7-5929e97c5792 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:27.627953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:27.645621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2fc04-a33f-43dd-b33b-b4e1f60b7497 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:27.654713 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:27.678428 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf85a39-70dc-48be-a9f8-2564c476493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:27.679293 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf85a39-70dc-48be-a9f8-2564c476493c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:27.679565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf85a39-70dc-48be-a9f8-2564c476493c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:27.679861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf85a39-70dc-48be-a9f8-2564c476493c None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:27.680578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf85a39-70dc-48be-a9f8-2564c476493c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:27.680881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf85a39-70dc-48be-a9f8-2564c476493c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:27.682296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf85a39-70dc-48be-a9f8-2564c476493c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:27.699910 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 561/1098] 10.222.0.15 () {56 vars in 1133 bytes} [Wed Sep 7 16:05:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:27.705692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf85a39-70dc-48be-a9f8-2564c476493c None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:27.715810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c659cf1-8cdf-44d5-b0b7-5929e97c5792 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:27.749209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb478dd-5d17-4ed0-a8a2-e4c41c68dcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:27.749826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb478dd-5d17-4ed0-a8a2-e4c41c68dcd1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:27.750199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb478dd-5d17-4ed0-a8a2-e4c41c68dcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:27.750364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb478dd-5d17-4ed0-a8a2-e4c41c68dcd1 None admin] PATH_INFO: `/v3/projects/e125dbb56cc445bbb4360578df050214` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:27.751459 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb478dd-5d17-4ed0-a8a2-e4c41c68dcd1 None admin] RBAC: Authorizing `identity:get_project(project_id=e125dbb56cc445bbb4360578df050214)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:27.752902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb478dd-5d17-4ed0-a8a2-e4c41c68dcd1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:27.754506 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 538/1099] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:05:27 2022] GET /identity/v3/projects/e125dbb56cc445bbb4360578df050214 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:27.761409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb478dd-5d17-4ed0-a8a2-e4c41c68dcd1 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:27.803084 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf85a39-70dc-48be-a9f8-2564c476493c None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:27.837510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e03256-b66f-46ca-b3b2-da4c72bba0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:27.838110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03256-b66f-46ca-b3b2-da4c72bba0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:27.838385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03256-b66f-46ca-b3b2-da4c72bba0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:27.838679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e03256-b66f-46ca-b3b2-da4c72bba0b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:27.848633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:27.858032 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb478dd-5d17-4ed0-a8a2-e4c41c68dcd1 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:27.869264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:27.894586 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d37fde-f2c1-43cb-bf3d-e82ed7e2f744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:27.895611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d37fde-f2c1-43cb-bf3d-e82ed7e2f744 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:27.896149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d37fde-f2c1-43cb-bf3d-e82ed7e2f744 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:27.896615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d37fde-f2c1-43cb-bf3d-e82ed7e2f744 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:27.897921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d37fde-f2c1-43cb-bf3d-e82ed7e2f744 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=e125dbb56cc445bbb4360578df050214, user.id=b076537b32ee409ca57da121141e3efc) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:27.898449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d37fde-f2c1-43cb-bf3d-e82ed7e2f744 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:27.900167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d37fde-f2c1-43cb-bf3d-e82ed7e2f744 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:27.906781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e03256-b66f-46ca-b3b2-da4c72bba0b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:27.910938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 539/1100] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:05:27 2022] GET /identity/v3/role_assignments?user.id=b076537b32ee409ca57da121141e3efc&scope.project.id=e125dbb56cc445bbb4360578df050214&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:27.911117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e03256-b66f-46ca-b3b2-da4c72bba0b6 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:28.020362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 562/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:05:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 16:05:28.026374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d37fde-f2c1-43cb-bf3d-e82ed7e2f744 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:28.129599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d37fde-f2c1-43cb-bf3d-e82ed7e2f744 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:28.164096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208323d-4363-49f7-99b0-7064aa15be3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:28.164721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208323d-4363-49f7-99b0-7064aa15be3a None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:28.164999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208323d-4363-49f7-99b0-7064aa15be3a None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:28.165270 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208323d-4363-49f7-99b0-7064aa15be3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:28.175920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208323d-4363-49f7-99b0-7064aa15be3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:28.177350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208323d-4363-49f7-99b0-7064aa15be3a None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-b208323d-4363-49f7-99b0-7064aa15be3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:28.192162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:28.193093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:05:28.193994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:28.193994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:05:28.193994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:28.193994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 540/1102] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:28.199005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e03256-b66f-46ca-b3b2-da4c72bba0b6 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:28.243387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:28.301851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e03256-b66f-46ca-b3b2-da4c72bba0b6 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:28.335217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666cdda-e932-4af9-b8aa-05e75bcd8f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:28.335840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666cdda-e932-4af9-b8aa-05e75bcd8f5a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:28.336120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666cdda-e932-4af9-b8aa-05e75bcd8f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:28.336394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666cdda-e932-4af9-b8aa-05e75bcd8f5a None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:28.337024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666cdda-e932-4af9-b8aa-05e75bcd8f5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:28.337350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666cdda-e932-4af9-b8aa-05e75bcd8f5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:28.339210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666cdda-e932-4af9-b8aa-05e75bcd8f5a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:28.350285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 563/1103] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:28.356659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208323d-4363-49f7-99b0-7064aa15be3a None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:28.383779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:28.455173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208323d-4363-49f7-99b0-7064aa15be3a None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:28.488656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716b65c-8e48-448e-a4d0-6d1cb7044bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:28.489279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716b65c-8e48-448e-a4d0-6d1cb7044bfd None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:28.489557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716b65c-8e48-448e-a4d0-6d1cb7044bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:28.489831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716b65c-8e48-448e-a4d0-6d1cb7044bfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:28.496417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716b65c-8e48-448e-a4d0-6d1cb7044bfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:28.497843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716b65c-8e48-448e-a4d0-6d1cb7044bfd None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-9716b65c-8e48-448e-a4d0-6d1cb7044bfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:05:28.508942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:28.510018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:28.511456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:05:28.511456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:05:28.511456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:05:28.511456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:05:28.511456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:05:28.511456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:05:28.511456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:05:28.511456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:28.511456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 541/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:05:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:28.516458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666cdda-e932-4af9-b8aa-05e75bcd8f5a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:28.616538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:28.619508 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666cdda-e932-4af9-b8aa-05e75bcd8f5a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:28.653642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612bdea8-712d-4a34-a58a-717cddadad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:28.654188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bdea8-712d-4a34-a58a-717cddadad89 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:28.654454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bdea8-712d-4a34-a58a-717cddadad89 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:28.654983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612bdea8-712d-4a34-a58a-717cddadad89 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:28.655851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bdea8-712d-4a34-a58a-717cddadad89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:28.656548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bdea8-712d-4a34-a58a-717cddadad89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:28.657972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612bdea8-712d-4a34-a58a-717cddadad89 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:28.669500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 564/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:05:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:28.675425 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716b65c-8e48-448e-a4d0-6d1cb7044bfd None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:28.756954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:28.771195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716b65c-8e48-448e-a4d0-6d1cb7044bfd None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:28.804759 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa4d94-57fe-48be-8ef7-9b24643e50f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:28.805379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa4d94-57fe-48be-8ef7-9b24643e50f1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:28.805650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa4d94-57fe-48be-8ef7-9b24643e50f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:28.805916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa4d94-57fe-48be-8ef7-9b24643e50f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:28.806646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa4d94-57fe-48be-8ef7-9b24643e50f1 None admin] RBAC: Adding query filter params (role.id=2c8c5f15b3624270a26334ed3ebdf44a, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=45061db68b1e40f59935cc9a46d2ff10) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:28.806949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa4d94-57fe-48be-8ef7-9b24643e50f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:28.808517 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa4d94-57fe-48be-8ef7-9b24643e50f1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:28.815065 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 542/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:05:28 2022] GET /identity/v3/role_assignments?user.id=45061db68b1e40f59935cc9a46d2ff10&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=2c8c5f15b3624270a26334ed3ebdf44a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:05:30.073978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2fcc4-bb89-4193-a750-45018308ef08 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:30.074474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2fcc4-bb89-4193-a750-45018308ef08 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:30.074580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2fcc4-bb89-4193-a750-45018308ef08 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:30.075903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 565/1107] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:30.081080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3b457-55b6-4e07-847c-1cd30a17402b None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:30.081461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3b457-55b6-4e07-847c-1cd30a17402b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:30.081628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3b457-55b6-4e07-847c-1cd30a17402b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:30.099245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-23f3b457-55b6-4e07-847c-1cd30a17402b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:30.100442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-23f3b457-55b6-4e07-847c-1cd30a17402b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:30.163874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 543/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:30.433704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2fcc4-bb89-4193-a750-45018308ef08 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:30.532246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2fcc4-bb89-4193-a750-45018308ef08 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:30.565401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98a644-fa11-45c0-9ea8-72401da7baf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:30.566013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98a644-fa11-45c0-9ea8-72401da7baf7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:30.566283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98a644-fa11-45c0-9ea8-72401da7baf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:30.566549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98a644-fa11-45c0-9ea8-72401da7baf7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:30.567628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98a644-fa11-45c0-9ea8-72401da7baf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:30.569102 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98a644-fa11-45c0-9ea8-72401da7baf7 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:30.570616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 566/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:30.576283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3b457-55b6-4e07-847c-1cd30a17402b None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:30.654853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:30.673843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3b457-55b6-4e07-847c-1cd30a17402b None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:30.706803 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1fd7b-821d-4194-aec0-b47328ee64d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:30.707333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1fd7b-821d-4194-aec0-b47328ee64d9 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:30.707607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1fd7b-821d-4194-aec0-b47328ee64d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:30.707902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1fd7b-821d-4194-aec0-b47328ee64d9 None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:30.708537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1fd7b-821d-4194-aec0-b47328ee64d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:30.709934 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1fd7b-821d-4194-aec0-b47328ee64d9 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:30.801847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 544/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:05:30 2022] POST /identity/v3/users => generated 286 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:05:32.993468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e73551-882a-4af0-b918-05d15e014c55 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:32.994022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e73551-882a-4af0-b918-05d15e014c55 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:32.994022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e73551-882a-4af0-b918-05d15e014c55 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:32.995917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 567/1111] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:33.001407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34250b-c8c8-4c7b-a032-436125b1b89c None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:33.001834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34250b-c8c8-4c7b-a032-436125b1b89c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:33.010923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34250b-c8c8-4c7b-a032-436125b1b89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:33.026413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-9c34250b-c8c8-4c7b-a032-436125b1b89c None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:33.027592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9c34250b-c8c8-4c7b-a032-436125b1b89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:33.092811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 545/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:33.362923 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e73551-882a-4af0-b918-05d15e014c55 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:33.390527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:33.460989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e73551-882a-4af0-b918-05d15e014c55 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:33.493524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6b59-c1df-43bf-bd6e-61b134a50cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:33.494131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6b59-c1df-43bf-bd6e-61b134a50cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:33.494407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6b59-c1df-43bf-bd6e-61b134a50cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:33.494711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6b59-c1df-43bf-bd6e-61b134a50cc5 None admin] PATH_INFO: `/v3/roles/f8c4be33fa2f4e69a3de71a3c4d7b179` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:33.496094 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6b59-c1df-43bf-bd6e-61b134a50cc5 None admin] RBAC: Authorizing `identity:get_role(role_id=f8c4be33fa2f4e69a3de71a3c4d7b179)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:33.497544 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6b59-c1df-43bf-bd6e-61b134a50cc5 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:33.498814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 568/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:33 2022] GET /identity/v3/roles/f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:33.504656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34250b-c8c8-4c7b-a032-436125b1b89c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:33.568476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:33.601695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34250b-c8c8-4c7b-a032-436125b1b89c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:33.634625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124fee3-0158-46f4-a80c-87cca72c8943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:33.635190 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124fee3-0158-46f4-a80c-87cca72c8943 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:33.635469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124fee3-0158-46f4-a80c-87cca72c8943 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:33.635742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124fee3-0158-46f4-a80c-87cca72c8943 None admin] PATH_INFO: `/v3/users/38fc38412f054cd8b6b0b042971a96b2` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:33.648330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124fee3-0158-46f4-a80c-87cca72c8943 None admin] RBAC: Authorizing `identity:get_user(user_id=38fc38412f054cd8b6b0b042971a96b2)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:33.649750 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124fee3-0158-46f4-a80c-87cca72c8943 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:33.651842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 546/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:33 2022] GET /identity/v3/users/38fc38412f054cd8b6b0b042971a96b2 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:33.657529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6b59-c1df-43bf-bd6e-61b134a50cc5 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:33.747445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:33.752941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6b59-c1df-43bf-bd6e-61b134a50cc5 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:33.785829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5f254-451e-46db-8a36-c02fee18887b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:33.786410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5f254-451e-46db-8a36-c02fee18887b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:33.786685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5f254-451e-46db-8a36-c02fee18887b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:33.786952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5f254-451e-46db-8a36-c02fee18887b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:33.850623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5f254-451e-46db-8a36-c02fee18887b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:33.852058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca5f254-451e-46db-8a36-c02fee18887b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:33.938572 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:33.944575 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 569/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:05:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:05:33.953086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124fee3-0158-46f4-a80c-87cca72c8943 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:34.049945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124fee3-0158-46f4-a80c-87cca72c8943 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:34.073270 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:34.083519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823771c-4647-48a7-ac95-7be2b74e3240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:34.084262 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823771c-4647-48a7-ac95-7be2b74e3240 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:34.084612 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823771c-4647-48a7-ac95-7be2b74e3240 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:34.084891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823771c-4647-48a7-ac95-7be2b74e3240 None admin] PATH_INFO: `/v3/projects/e125dbb56cc445bbb4360578df050214` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:34.085994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823771c-4647-48a7-ac95-7be2b74e3240 None admin] RBAC: Authorizing `identity:get_project(project_id=e125dbb56cc445bbb4360578df050214)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:34.087412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4823771c-4647-48a7-ac95-7be2b74e3240 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:34.089980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 547/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:05:33 2022] GET /identity/v3/projects/e125dbb56cc445bbb4360578df050214 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:34.096275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5f254-451e-46db-8a36-c02fee18887b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:34.193395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5f254-451e-46db-8a36-c02fee18887b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:34.225766 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8321f4-0b19-4695-a178-9e8be7045e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:34.226352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8321f4-0b19-4695-a178-9e8be7045e86 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:34.226627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8321f4-0b19-4695-a178-9e8be7045e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:34.226897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8321f4-0b19-4695-a178-9e8be7045e86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:34.227605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8321f4-0b19-4695-a178-9e8be7045e86 None admin] RBAC: Adding query filter params (role.id=f8c4be33fa2f4e69a3de71a3c4d7b179, scope.project.id=e125dbb56cc445bbb4360578df050214, user.id=38fc38412f054cd8b6b0b042971a96b2) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:34.227941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8321f4-0b19-4695-a178-9e8be7045e86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:34.229346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8321f4-0b19-4695-a178-9e8be7045e86 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:34.234476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 570/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:05:34 2022] GET /identity/v3/role_assignments?user.id=38fc38412f054cd8b6b0b042971a96b2&scope.project.id=e125dbb56cc445bbb4360578df050214&role.id=f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:36.318480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a3b16-7562-4578-b63a-afe285ebf064 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:36.318990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a3b16-7562-4578-b63a-afe285ebf064 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:36.318990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a3b16-7562-4578-b63a-afe285ebf064 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:36.320309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 548/1118] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:36.325576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a88ca9-a58a-47dd-b5ea-6f10d2ba0eda None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:36.326006 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a88ca9-a58a-47dd-b5ea-6f10d2ba0eda None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:36.326126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a88ca9-a58a-47dd-b5ea-6f10d2ba0eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:36.343814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-49a88ca9-a58a-47dd-b5ea-6f10d2ba0eda None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:36.345010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-49a88ca9-a58a-47dd-b5ea-6f10d2ba0eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:36.397908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:36.408934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 571/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:36.678243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a3b16-7562-4578-b63a-afe285ebf064 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:36.777879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a3b16-7562-4578-b63a-afe285ebf064 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:36.810231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d6bc1-f34a-4f7f-86a2-35a1877cd15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:36.810846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6bc1-f34a-4f7f-86a2-35a1877cd15d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:36.811127 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6bc1-f34a-4f7f-86a2-35a1877cd15d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:36.811404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d6bc1-f34a-4f7f-86a2-35a1877cd15d None admin] PATH_INFO: `/v3/roles/f8c4be33fa2f4e69a3de71a3c4d7b179` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:36.812806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d6bc1-f34a-4f7f-86a2-35a1877cd15d None admin] RBAC: Authorizing `identity:get_role(role_id=f8c4be33fa2f4e69a3de71a3c4d7b179)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:36.814164 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d6bc1-f34a-4f7f-86a2-35a1877cd15d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:36.815449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 549/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:36 2022] GET /identity/v3/roles/f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:36.820916 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a88ca9-a58a-47dd-b5ea-6f10d2ba0eda None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:36.915975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a88ca9-a58a-47dd-b5ea-6f10d2ba0eda None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:36.948546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11989c44-09b8-4668-ba29-715142a80681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:36.949107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11989c44-09b8-4668-ba29-715142a80681 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:36.949379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11989c44-09b8-4668-ba29-715142a80681 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:36.949649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11989c44-09b8-4668-ba29-715142a80681 None admin] PATH_INFO: `/v3/users/38fc38412f054cd8b6b0b042971a96b2` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:36.950730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11989c44-09b8-4668-ba29-715142a80681 None admin] RBAC: Authorizing `identity:get_user(user_id=38fc38412f054cd8b6b0b042971a96b2)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:36.952173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11989c44-09b8-4668-ba29-715142a80681 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:36.953696 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 572/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:36 2022] GET /identity/v3/users/38fc38412f054cd8b6b0b042971a96b2 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:36.959268 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d6bc1-f34a-4f7f-86a2-35a1877cd15d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:37.017326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:37.056637 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d6bc1-f34a-4f7f-86a2-35a1877cd15d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:37.091006 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d1268-1a21-4200-a607-51e7cc44aa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:37.091615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d1268-1a21-4200-a607-51e7cc44aa1b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:37.091922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d1268-1a21-4200-a607-51e7cc44aa1b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:37.092194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d1268-1a21-4200-a607-51e7cc44aa1b None admin] PATH_INFO: `/v3/projects/e125dbb56cc445bbb4360578df050214` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:37.093284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d1268-1a21-4200-a607-51e7cc44aa1b None admin] RBAC: Authorizing `identity:get_project(project_id=e125dbb56cc445bbb4360578df050214)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:37.094700 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d1268-1a21-4200-a607-51e7cc44aa1b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:37.096330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 550/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:05:36 2022] GET /identity/v3/projects/e125dbb56cc445bbb4360578df050214 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:37.102386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11989c44-09b8-4668-ba29-715142a80681 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:37.163117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:37.203893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11989c44-09b8-4668-ba29-715142a80681 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:37.235931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08bce7-c318-403c-bf6d-712de36831fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:37.236573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08bce7-c318-403c-bf6d-712de36831fd None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:37.236849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08bce7-c318-403c-bf6d-712de36831fd None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:37.237120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08bce7-c318-403c-bf6d-712de36831fd None admin] PATH_INFO: `/v3/projects/e125dbb56cc445bbb4360578df050214/users/38fc38412f054cd8b6b0b042971a96b2/roles/f8c4be33fa2f4e69a3de71a3c4d7b179` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:37.239193 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08bce7-c318-403c-bf6d-712de36831fd None admin] RBAC: Authorizing `identity:create_grant(project_id=e125dbb56cc445bbb4360578df050214, user_id=38fc38412f054cd8b6b0b042971a96b2, role_id=f8c4be33fa2f4e69a3de71a3c4d7b179)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:37.240635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08bce7-c318-403c-bf6d-712de36831fd None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:37.306253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 573/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:05:37 2022] PUT /identity/v3/projects/e125dbb56cc445bbb4360578df050214/users/38fc38412f054cd8b6b0b042971a96b2/roles/f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:05:39.414118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38d913-dd08-4bd2-af2a-1abb25af039b None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:39.414626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38d913-dd08-4bd2-af2a-1abb25af039b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:39.414710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38d913-dd08-4bd2-af2a-1abb25af039b None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:39.416061 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 551/1124] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:39.421254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fd0ba-c875-4ef5-ba46-3b200cb673a2 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:39.421682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fd0ba-c875-4ef5-ba46-3b200cb673a2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:39.421801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fd0ba-c875-4ef5-ba46-3b200cb673a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:39.440328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1b9fd0ba-c875-4ef5-ba46-3b200cb673a2 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:39.441507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1b9fd0ba-c875-4ef5-ba46-3b200cb673a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:39.505358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 574/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:39.772826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38d913-dd08-4bd2-af2a-1abb25af039b None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:39.871807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38d913-dd08-4bd2-af2a-1abb25af039b None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:39.896976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:39.904320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fec0a-4d2b-4dc5-9ad1-077750a42864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:39.904940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fec0a-4d2b-4dc5-9ad1-077750a42864 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:39.905218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fec0a-4d2b-4dc5-9ad1-077750a42864 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:39.905492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fec0a-4d2b-4dc5-9ad1-077750a42864 None admin] PATH_INFO: `/v3/roles/f8c4be33fa2f4e69a3de71a3c4d7b179` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:39.906871 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fec0a-4d2b-4dc5-9ad1-077750a42864 None admin] RBAC: Authorizing `identity:get_role(role_id=f8c4be33fa2f4e69a3de71a3c4d7b179)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:39.908267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fec0a-4d2b-4dc5-9ad1-077750a42864 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:39.909565 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 552/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:39 2022] GET /identity/v3/roles/f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:39.919979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fd0ba-c875-4ef5-ba46-3b200cb673a2 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:39.962901 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:40.011621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fd0ba-c875-4ef5-ba46-3b200cb673a2 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:40.045255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93549fcb-2b28-45e0-adf3-939c50f15b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:40.045817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549fcb-2b28-45e0-adf3-939c50f15b70 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:40.046096 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549fcb-2b28-45e0-adf3-939c50f15b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:40.046370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93549fcb-2b28-45e0-adf3-939c50f15b70 None admin] PATH_INFO: `/v3/users/38fc38412f054cd8b6b0b042971a96b2` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:40.047475 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93549fcb-2b28-45e0-adf3-939c50f15b70 None admin] RBAC: Authorizing `identity:get_user(user_id=38fc38412f054cd8b6b0b042971a96b2)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:40.048950 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93549fcb-2b28-45e0-adf3-939c50f15b70 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:40.050516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 575/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:39 2022] GET /identity/v3/users/38fc38412f054cd8b6b0b042971a96b2 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:40.057237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fec0a-4d2b-4dc5-9ad1-077750a42864 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:40.153178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fec0a-4d2b-4dc5-9ad1-077750a42864 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:40.186247 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f9893-ea24-4e70-9696-0978876862b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:40.186836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f9893-ea24-4e70-9696-0978876862b4 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:40.187118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f9893-ea24-4e70-9696-0978876862b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:40.187384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f9893-ea24-4e70-9696-0978876862b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:40.245997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:40.252123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f9893-ea24-4e70-9696-0978876862b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:40.253542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101f9893-ea24-4e70-9696-0978876862b4 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:40.347930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 553/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:05:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:05:40.353654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93549fcb-2b28-45e0-adf3-939c50f15b70 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:40.449975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93549fcb-2b28-45e0-adf3-939c50f15b70 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:40.465569 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:40.482073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4dcb8-f8d6-496f-afe2-c20be18bcfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:40.482712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4dcb8-f8d6-496f-afe2-c20be18bcfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:40.482991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4dcb8-f8d6-496f-afe2-c20be18bcfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:40.483264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4dcb8-f8d6-496f-afe2-c20be18bcfc8 None admin] PATH_INFO: `/v3/projects/e125dbb56cc445bbb4360578df050214` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:40.484386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4dcb8-f8d6-496f-afe2-c20be18bcfc8 None admin] RBAC: Authorizing `identity:get_project(project_id=e125dbb56cc445bbb4360578df050214)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:40.485802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4dcb8-f8d6-496f-afe2-c20be18bcfc8 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:40.487381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 576/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:05:40 2022] GET /identity/v3/projects/e125dbb56cc445bbb4360578df050214 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:40.493481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f9893-ea24-4e70-9696-0978876862b4 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:40.584404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:40.589920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f9893-ea24-4e70-9696-0978876862b4 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:40.623035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29dc2fd-7d16-4aea-8f84-850849987f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:40.623628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dc2fd-7d16-4aea-8f84-850849987f1d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:40.623931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dc2fd-7d16-4aea-8f84-850849987f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:40.624202 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dc2fd-7d16-4aea-8f84-850849987f1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:40.624911 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29dc2fd-7d16-4aea-8f84-850849987f1d None admin] RBAC: Adding query filter params (role.id=f8c4be33fa2f4e69a3de71a3c4d7b179, scope.project.id=e125dbb56cc445bbb4360578df050214, user.id=38fc38412f054cd8b6b0b042971a96b2) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:40.625213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29dc2fd-7d16-4aea-8f84-850849987f1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:40.626614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29dc2fd-7d16-4aea-8f84-850849987f1d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:40.632211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 554/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:05:40 2022] GET /identity/v3/role_assignments?user.id=38fc38412f054cd8b6b0b042971a96b2&scope.project.id=e125dbb56cc445bbb4360578df050214&role.id=f8c4be33fa2f4e69a3de71a3c4d7b179 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:42.794783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edef64-c69c-45ad-972d-1a7080b9fec0 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:42.795308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edef64-c69c-45ad-972d-1a7080b9fec0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:42.795308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79edef64-c69c-45ad-972d-1a7080b9fec0 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:42.796702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 577/1131] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:42.802021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533fe11-025e-4940-a25a-93e34c3bacd1 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:42.802408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533fe11-025e-4940-a25a-93e34c3bacd1 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:42.802580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3533fe11-025e-4940-a25a-93e34c3bacd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:42.821379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-3533fe11-025e-4940-a25a-93e34c3bacd1 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:42.822567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3533fe11-025e-4940-a25a-93e34c3bacd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:42.886851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 555/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:43.159755 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edef64-c69c-45ad-972d-1a7080b9fec0 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:43.256774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79edef64-c69c-45ad-972d-1a7080b9fec0 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:43.288964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3576fd06-c536-4567-9837-bfb3467c4a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:43.289576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576fd06-c536-4567-9837-bfb3467c4a2c None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:43.289850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576fd06-c536-4567-9837-bfb3467c4a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:43.290117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3576fd06-c536-4567-9837-bfb3467c4a2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:43.291736 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576fd06-c536-4567-9837-bfb3467c4a2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:43.293180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3576fd06-c536-4567-9837-bfb3467c4a2c None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:43.294700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 578/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:43.300480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533fe11-025e-4940-a25a-93e34c3bacd1 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:43.390553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:43.399359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3533fe11-025e-4940-a25a-93e34c3bacd1 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:43.433068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d71c2d-04d3-4986-aae4-7a1c641322e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:43.433602 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d71c2d-04d3-4986-aae4-7a1c641322e5 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:43.433877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d71c2d-04d3-4986-aae4-7a1c641322e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:43.434145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d71c2d-04d3-4986-aae4-7a1c641322e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:43.434793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d71c2d-04d3-4986-aae4-7a1c641322e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:43.436236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d71c2d-04d3-4986-aae4-7a1c641322e5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:43.513334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 556/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:05:43 2022] POST /identity/v3/projects => generated 311 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:05:45.662103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd01f5-8c26-4859-bae5-d0d26875afbc None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:45.662627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd01f5-8c26-4859-bae5-d0d26875afbc None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:45.662742 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd01f5-8c26-4859-bae5-d0d26875afbc None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:45.664019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 579/1135] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:45.669702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819f1d8-7ad1-49de-ab4e-6890ec7774f8 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:45.670137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819f1d8-7ad1-49de-ab4e-6890ec7774f8 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:45.670237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819f1d8-7ad1-49de-ab4e-6890ec7774f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:45.688330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-1819f1d8-7ad1-49de-ab4e-6890ec7774f8 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:45.689507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1819f1d8-7ad1-49de-ab4e-6890ec7774f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:45.752918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 557/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:05:46.021497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd01f5-8c26-4859-bae5-d0d26875afbc None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:46.075398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:46.121257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd01f5-8c26-4859-bae5-d0d26875afbc None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:46.154342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea3ff25-3aa2-45ad-9f15-c19a21903e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:46.155013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3ff25-3aa2-45ad-9f15-c19a21903e79 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:46.155289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3ff25-3aa2-45ad-9f15-c19a21903e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:46.155561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea3ff25-3aa2-45ad-9f15-c19a21903e79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:46.156679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3ff25-3aa2-45ad-9f15-c19a21903e79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:46.158092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea3ff25-3aa2-45ad-9f15-c19a21903e79 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:46.159622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 580/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:05:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:46.165370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819f1d8-7ad1-49de-ab4e-6890ec7774f8 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:46.232921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:46.261166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819f1d8-7ad1-49de-ab4e-6890ec7774f8 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:46.293079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c602e3-f6d4-4ce8-a976-b78a2f46bf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:46.293763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c602e3-f6d4-4ce8-a976-b78a2f46bf7e None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:46.294044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c602e3-f6d4-4ce8-a976-b78a2f46bf7e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:46.294314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c602e3-f6d4-4ce8-a976-b78a2f46bf7e None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:46.294964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c602e3-f6d4-4ce8-a976-b78a2f46bf7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:46.296398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c602e3-f6d4-4ce8-a976-b78a2f46bf7e None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:46.401292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 558/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:05:46 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:05:48.551432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5438a2-0104-4a6c-a363-8671adab8d45 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:48.552011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5438a2-0104-4a6c-a363-8671adab8d45 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:48.552011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5438a2-0104-4a6c-a363-8671adab8d45 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:48.553318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 581/1139] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:48.558664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c33845-9cff-4872-b8a8-b3ddf0e9e680 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:48.559085 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c33845-9cff-4872-b8a8-b3ddf0e9e680 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:48.559236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c33845-9cff-4872-b8a8-b3ddf0e9e680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:48.577148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-e9c33845-9cff-4872-b8a8-b3ddf0e9e680 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:48.578329 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e9c33845-9cff-4872-b8a8-b3ddf0e9e680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:48.643316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 559/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:48.909915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5438a2-0104-4a6c-a363-8671adab8d45 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:49.009895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5438a2-0104-4a6c-a363-8671adab8d45 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:49.029392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:49.042972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54d577-778b-4216-b093-fd94c0793138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:49.043601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54d577-778b-4216-b093-fd94c0793138 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:49.043907 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54d577-778b-4216-b093-fd94c0793138 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:49.044185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54d577-778b-4216-b093-fd94c0793138 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:49.054026 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e54d577-778b-4216-b093-fd94c0793138 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-4e54d577-778b-4216-b093-fd94c0793138 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:05:49.059490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:49.060523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:49.061402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 582/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:05:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:49.066645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c33845-9cff-4872-b8a8-b3ddf0e9e680 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:49.128851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:49.169418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c33845-9cff-4872-b8a8-b3ddf0e9e680 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:49.202043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5fa0f-81f3-495e-9147-c21f804351cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:49.202638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5fa0f-81f3-495e-9147-c21f804351cf None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:49.202911 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5fa0f-81f3-495e-9147-c21f804351cf None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:49.203178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5fa0f-81f3-495e-9147-c21f804351cf None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:49.203849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5fa0f-81f3-495e-9147-c21f804351cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:49.204147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5fa0f-81f3-495e-9147-c21f804351cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:49.205493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5fa0f-81f3-495e-9147-c21f804351cf None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:49.213705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 560/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:05:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:49.219652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54d577-778b-4216-b093-fd94c0793138 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:49.317712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54d577-778b-4216-b093-fd94c0793138 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:49.350498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25068cb4-ff46-42ad-8a56-ac1ab0750da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:49.351106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25068cb4-ff46-42ad-8a56-ac1ab0750da1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:49.351393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25068cb4-ff46-42ad-8a56-ac1ab0750da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:49.351665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25068cb4-ff46-42ad-8a56-ac1ab0750da1 None admin] PATH_INFO: `/v3/users/965564929a0c4840aa189bc1d9f77950` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:49.364806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25068cb4-ff46-42ad-8a56-ac1ab0750da1 None admin] RBAC: Authorizing `identity:get_user(user_id=965564929a0c4840aa189bc1d9f77950)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:49.366241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25068cb4-ff46-42ad-8a56-ac1ab0750da1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:49.367812 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 583/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:49 2022] GET /identity/v3/users/965564929a0c4840aa189bc1d9f77950 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:49.373457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5fa0f-81f3-495e-9147-c21f804351cf None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:49.469217 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5fa0f-81f3-495e-9147-c21f804351cf None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:49.488269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:49.501726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3ecf9-29fb-4278-a3ee-cc4c2e8e4ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:49.502322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3ecf9-29fb-4278-a3ee-cc4c2e8e4ffc None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:49.502597 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3ecf9-29fb-4278-a3ee-cc4c2e8e4ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:49.502868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c3ecf9-29fb-4278-a3ee-cc4c2e8e4ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:49.567550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3ecf9-29fb-4278-a3ee-cc4c2e8e4ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:49.569001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c3ecf9-29fb-4278-a3ee-cc4c2e8e4ffc None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:49.664399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 561/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:05:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:05:49.670823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25068cb4-ff46-42ad-8a56-ac1ab0750da1 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:49.709725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:49.767240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25068cb4-ff46-42ad-8a56-ac1ab0750da1 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:49.799755 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b25cd-c410-41d2-b12a-eeb1ae43232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:49.800389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b25cd-c410-41d2-b12a-eeb1ae43232e None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:49.800667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b25cd-c410-41d2-b12a-eeb1ae43232e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:49.800939 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b25cd-c410-41d2-b12a-eeb1ae43232e None admin] PATH_INFO: `/v3/projects/91c47e53385945fe8ae9b88c0f0a228b` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:49.802028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b25cd-c410-41d2-b12a-eeb1ae43232e None admin] RBAC: Authorizing `identity:get_project(project_id=91c47e53385945fe8ae9b88c0f0a228b)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:49.803703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b25cd-c410-41d2-b12a-eeb1ae43232e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:49.805324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 584/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:05:49 2022] GET /identity/v3/projects/91c47e53385945fe8ae9b88c0f0a228b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:49.811267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3ecf9-29fb-4278-a3ee-cc4c2e8e4ffc None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:49.831840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:49.907335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c3ecf9-29fb-4278-a3ee-cc4c2e8e4ffc None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:49.940263 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45829cd2-5636-41ee-a84e-99bfc8329392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:49.940864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45829cd2-5636-41ee-a84e-99bfc8329392 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:49.941144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45829cd2-5636-41ee-a84e-99bfc8329392 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:49.941419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45829cd2-5636-41ee-a84e-99bfc8329392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:49.942139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45829cd2-5636-41ee-a84e-99bfc8329392 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=91c47e53385945fe8ae9b88c0f0a228b, user.id=965564929a0c4840aa189bc1d9f77950) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:49.942449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45829cd2-5636-41ee-a84e-99bfc8329392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:49.943921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45829cd2-5636-41ee-a84e-99bfc8329392 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:49.949815 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 562/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:05:49 2022] GET /identity/v3/role_assignments?user.id=965564929a0c4840aa189bc1d9f77950&scope.project.id=91c47e53385945fe8ae9b88c0f0a228b&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:52.045491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a0944-a42d-48d7-8da7-081cd22f7130 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:52.045945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a0944-a42d-48d7-8da7-081cd22f7130 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:52.046035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a0944-a42d-48d7-8da7-081cd22f7130 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:52.047361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 585/1147] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:52.052600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6521-926d-4057-9655-fdfb9e69ab26 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:52.053020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6521-926d-4057-9655-fdfb9e69ab26 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:52.053148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6521-926d-4057-9655-fdfb9e69ab26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:52.071584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-f8dc6521-926d-4057-9655-fdfb9e69ab26 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:52.072788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f8dc6521-926d-4057-9655-fdfb9e69ab26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:52.136959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 563/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 16:05:52.402152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a0944-a42d-48d7-8da7-081cd22f7130 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:52.499103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a0944-a42d-48d7-8da7-081cd22f7130 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:52.506040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:52.531944 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf986dbe-0ffe-496a-ae27-50a22d615c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:52.532557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf986dbe-0ffe-496a-ae27-50a22d615c91 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:52.532829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf986dbe-0ffe-496a-ae27-50a22d615c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:52.533137 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf986dbe-0ffe-496a-ae27-50a22d615c91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:52.542892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf986dbe-0ffe-496a-ae27-50a22d615c91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-cf986dbe-0ffe-496a-ae27-50a22d615c91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:05:52.548118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:52.549202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:52.550361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 586/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:05:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:52.555603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc6521-926d-4057-9655-fdfb9e69ab26 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:52.570063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:52.650698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc6521-926d-4057-9655-fdfb9e69ab26 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:52.682686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75612452-8291-4782-9d97-381306a6c2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:52.683219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75612452-8291-4782-9d97-381306a6c2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:52.683493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75612452-8291-4782-9d97-381306a6c2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:52.683784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75612452-8291-4782-9d97-381306a6c2a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:52.684445 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75612452-8291-4782-9d97-381306a6c2a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:52.684744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75612452-8291-4782-9d97-381306a6c2a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:52.686168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75612452-8291-4782-9d97-381306a6c2a3 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:52.694617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 564/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:05:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:52.701170 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf986dbe-0ffe-496a-ae27-50a22d615c91 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:52.799461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf986dbe-0ffe-496a-ae27-50a22d615c91 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:52.831842 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e45fb-2932-46da-be79-42186ee07b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:52.832456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e45fb-2932-46da-be79-42186ee07b85 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:52.832737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e45fb-2932-46da-be79-42186ee07b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:52.833010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e45fb-2932-46da-be79-42186ee07b85 None admin] PATH_INFO: `/v3/users/965564929a0c4840aa189bc1d9f77950` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:52.834107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e45fb-2932-46da-be79-42186ee07b85 None admin] RBAC: Authorizing `identity:get_user(user_id=965564929a0c4840aa189bc1d9f77950)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:52.835519 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e45fb-2932-46da-be79-42186ee07b85 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:52.837637 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 587/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:52 2022] GET /identity/v3/users/965564929a0c4840aa189bc1d9f77950 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:52.843234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75612452-8291-4782-9d97-381306a6c2a3 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:52.916590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:52.939244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75612452-8291-4782-9d97-381306a6c2a3 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:52.975888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569862d-c9c9-4664-a33e-4d40773058f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:52.976515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569862d-c9c9-4664-a33e-4d40773058f1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:52.976793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569862d-c9c9-4664-a33e-4d40773058f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:52.977067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569862d-c9c9-4664-a33e-4d40773058f1 None admin] PATH_INFO: `/v3/projects/91c47e53385945fe8ae9b88c0f0a228b` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:52.978187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569862d-c9c9-4664-a33e-4d40773058f1 None admin] RBAC: Authorizing `identity:get_project(project_id=91c47e53385945fe8ae9b88c0f0a228b)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:52.979603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569862d-c9c9-4664-a33e-4d40773058f1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:52.981265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 565/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:05:52 2022] GET /identity/v3/projects/91c47e53385945fe8ae9b88c0f0a228b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:52.988557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e45fb-2932-46da-be79-42186ee07b85 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:53.023329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:53.087484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e45fb-2932-46da-be79-42186ee07b85 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:53.121025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bbdfa-4302-4d20-a3a1-64329067933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:53.121679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bbdfa-4302-4d20-a3a1-64329067933e None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:53.121960 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bbdfa-4302-4d20-a3a1-64329067933e None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:53.122236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bbdfa-4302-4d20-a3a1-64329067933e None admin] PATH_INFO: `/v3/projects/91c47e53385945fe8ae9b88c0f0a228b/users/965564929a0c4840aa189bc1d9f77950/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:53.124136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bbdfa-4302-4d20-a3a1-64329067933e None admin] RBAC: Authorizing `identity:create_grant(project_id=91c47e53385945fe8ae9b88c0f0a228b, user_id=965564929a0c4840aa189bc1d9f77950, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:53.125560 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bbdfa-4302-4d20-a3a1-64329067933e None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:53.223609 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 588/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:05:52 2022] PUT /identity/v3/projects/91c47e53385945fe8ae9b88c0f0a228b/users/965564929a0c4840aa189bc1d9f77950/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:05:55.335286 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123cd5f-6018-4a88-a7ac-7848f1034b1c None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:55.335827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123cd5f-6018-4a88-a7ac-7848f1034b1c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:55.335827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123cd5f-6018-4a88-a7ac-7848f1034b1c None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:55.337610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 566/1154] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:55.342682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396871c-01d4-4e30-bcd8-fde113c16c9a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:55.343097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396871c-01d4-4e30-bcd8-fde113c16c9a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:55.343234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396871c-01d4-4e30-bcd8-fde113c16c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:55.361609 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6396871c-01d4-4e30-bcd8-fde113c16c9a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:55.362792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6396871c-01d4-4e30-bcd8-fde113c16c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:55.426817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 589/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:55.693697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123cd5f-6018-4a88-a7ac-7848f1034b1c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:55.794130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123cd5f-6018-4a88-a7ac-7848f1034b1c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:55.826563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b4a69-e8c5-49b3-b2cc-180dac8f99a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:55.827175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b4a69-e8c5-49b3-b2cc-180dac8f99a7 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:55.827455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b4a69-e8c5-49b3-b2cc-180dac8f99a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:55.827730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b4a69-e8c5-49b3-b2cc-180dac8f99a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:55.838614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b4a69-e8c5-49b3-b2cc-180dac8f99a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-9d2b4a69-e8c5-49b3-b2cc-180dac8f99a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:05:55.843435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:55.844555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:05:55.845743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 567/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:05:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:55.850304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396871c-01d4-4e30-bcd8-fde113c16c9a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:55.873884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:55.945568 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396871c-01d4-4e30-bcd8-fde113c16c9a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:55.979436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e289a9-239f-4904-92b7-33900e1bc374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:55.980055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e289a9-239f-4904-92b7-33900e1bc374 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:55.980340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e289a9-239f-4904-92b7-33900e1bc374 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:55.980618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e289a9-239f-4904-92b7-33900e1bc374 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:55.981276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e289a9-239f-4904-92b7-33900e1bc374 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:55.981582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e289a9-239f-4904-92b7-33900e1bc374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:55.982949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e289a9-239f-4904-92b7-33900e1bc374 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:55.990732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 590/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:05:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:55.997229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b4a69-e8c5-49b3-b2cc-180dac8f99a7 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:56.000984 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:56.093717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b4a69-e8c5-49b3-b2cc-180dac8f99a7 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:56.125606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a049da6-c37e-41fe-9786-ec32e66757e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:56.126216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a049da6-c37e-41fe-9786-ec32e66757e5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:56.126490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a049da6-c37e-41fe-9786-ec32e66757e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:56.126761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a049da6-c37e-41fe-9786-ec32e66757e5 None admin] PATH_INFO: `/v3/users/965564929a0c4840aa189bc1d9f77950` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:56.127881 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a049da6-c37e-41fe-9786-ec32e66757e5 None admin] RBAC: Authorizing `identity:get_user(user_id=965564929a0c4840aa189bc1d9f77950)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:56.129291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a049da6-c37e-41fe-9786-ec32e66757e5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:56.130838 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 568/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:05:55 2022] GET /identity/v3/users/965564929a0c4840aa189bc1d9f77950 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:56.137156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e289a9-239f-4904-92b7-33900e1bc374 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:56.218297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:56.232141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e289a9-239f-4904-92b7-33900e1bc374 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:56.265190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e6dbc-31ca-4cdc-80b8-79de90222c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:56.265778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e6dbc-31ca-4cdc-80b8-79de90222c3a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:56.266055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e6dbc-31ca-4cdc-80b8-79de90222c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:56.266328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e6dbc-31ca-4cdc-80b8-79de90222c3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:56.332040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e6dbc-31ca-4cdc-80b8-79de90222c3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:56.333622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e6dbc-31ca-4cdc-80b8-79de90222c3a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:56.412825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:56.427880 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 591/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:05:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:05:56.434717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a049da6-c37e-41fe-9786-ec32e66757e5 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:56.502216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:56.529610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a049da6-c37e-41fe-9786-ec32e66757e5 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:56.562698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e423b9-7836-406a-8fde-3726b0e3be5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:56.563307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e423b9-7836-406a-8fde-3726b0e3be5e None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:56.563579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e423b9-7836-406a-8fde-3726b0e3be5e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:56.563872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e423b9-7836-406a-8fde-3726b0e3be5e None admin] PATH_INFO: `/v3/projects/91c47e53385945fe8ae9b88c0f0a228b` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:56.564960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e423b9-7836-406a-8fde-3726b0e3be5e None admin] RBAC: Authorizing `identity:get_project(project_id=91c47e53385945fe8ae9b88c0f0a228b)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:56.566360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e423b9-7836-406a-8fde-3726b0e3be5e None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:56.567973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 569/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:05:56 2022] GET /identity/v3/projects/91c47e53385945fe8ae9b88c0f0a228b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:56.574038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e6dbc-31ca-4cdc-80b8-79de90222c3a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:56.668525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e6dbc-31ca-4cdc-80b8-79de90222c3a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:56.700397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7dfea-766c-4a22-9d62-2187895d7cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:56.700978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7dfea-766c-4a22-9d62-2187895d7cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:56.701257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7dfea-766c-4a22-9d62-2187895d7cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:56.701527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7dfea-766c-4a22-9d62-2187895d7cc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:56.702232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7dfea-766c-4a22-9d62-2187895d7cc4 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=91c47e53385945fe8ae9b88c0f0a228b, user.id=965564929a0c4840aa189bc1d9f77950) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:56.702532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7dfea-766c-4a22-9d62-2187895d7cc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:56.703955 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7dfea-766c-4a22-9d62-2187895d7cc4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:56.709402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 592/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:05:56 2022] GET /identity/v3/role_assignments?user.id=965564929a0c4840aa189bc1d9f77950&scope.project.id=91c47e53385945fe8ae9b88c0f0a228b&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:05:58.880170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0a8e4-6093-428c-a849-681f41b7b5fe None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:58.880751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0a8e4-6093-428c-a849-681f41b7b5fe None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:58.880751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0a8e4-6093-428c-a849-681f41b7b5fe None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:58.882047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 570/1162] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:05:58.887241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb7a03-d5e0-4cb6-908f-24df7fe51109 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:58.887663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb7a03-d5e0-4cb6-908f-24df7fe51109 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:58.887794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb7a03-d5e0-4cb6-908f-24df7fe51109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:58.905905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-eeeb7a03-d5e0-4cb6-908f-24df7fe51109 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:58.907071 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-eeeb7a03-d5e0-4cb6-908f-24df7fe51109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:58.951193 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:58.971754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 593/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:05:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:05:59.248160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0a8e4-6093-428c-a849-681f41b7b5fe None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:59.358182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0a8e4-6093-428c-a849-681f41b7b5fe None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:59.404390 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf75d0-9e47-41bc-83d3-465632df4051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:59.405315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf75d0-9e47-41bc-83d3-465632df4051 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:59.405655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf75d0-9e47-41bc-83d3-465632df4051 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:59.405969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf75d0-9e47-41bc-83d3-465632df4051 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:59.417705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:59.473315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf75d0-9e47-41bc-83d3-465632df4051 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:59.474746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf75d0-9e47-41bc-83d3-465632df4051 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:59.567814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 571/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:05:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:05:59.574262 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb7a03-d5e0-4cb6-908f-24df7fe51109 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:59.669713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb7a03-d5e0-4cb6-908f-24df7fe51109 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:59.704933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ccb69-b8ea-4569-a511-fc8abf0f3bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:59.705558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ccb69-b8ea-4569-a511-fc8abf0f3bad None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:59.705836 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ccb69-b8ea-4569-a511-fc8abf0f3bad None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:59.706109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ccb69-b8ea-4569-a511-fc8abf0f3bad None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:59.712617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ccb69-b8ea-4569-a511-fc8abf0f3bad None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:59.714039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ccb69-b8ea-4569-a511-fc8abf0f3bad None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-b66ccb69-b8ea-4569-a511-fc8abf0f3bad None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:05:59.725125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:05:59.726401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:05:59.727474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 594/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 7 16:05:59 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:05:59.732254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf75d0-9e47-41bc-83d3-465632df4051 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:59.773676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:59.829000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf75d0-9e47-41bc-83d3-465632df4051 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:59.860823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbeb85-aa36-48f9-a2be-cf9895c49055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:59.861408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbeb85-aa36-48f9-a2be-cf9895c49055 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:59.861685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbeb85-aa36-48f9-a2be-cf9895c49055 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:59.861954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbeb85-aa36-48f9-a2be-cf9895c49055 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:59.862584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbeb85-aa36-48f9-a2be-cf9895c49055 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:05:59.862883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbeb85-aa36-48f9-a2be-cf9895c49055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:59.864258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cbeb85-aa36-48f9-a2be-cf9895c49055 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:59.871581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 572/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Wed Sep 7 16:05:59 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:01.974351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9bfe9-3bfa-4081-a289-a53329b0f704 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:01.974903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9bfe9-3bfa-4081-a289-a53329b0f704 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:01.974903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9bfe9-3bfa-4081-a289-a53329b0f704 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:01.977231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 595/1167] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:06:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:06:01.982526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11f89b-b541-4d18-bec4-412052512131 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:01.982936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11f89b-b541-4d18-bec4-412052512131 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:01.983066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11f89b-b541-4d18-bec4-412052512131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:02.000790 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-6b11f89b-b541-4d18-bec4-412052512131 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:06:02.001975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6b11f89b-b541-4d18-bec4-412052512131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:06:02.065754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 573/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:06:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:06:02.330047 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9bfe9-3bfa-4081-a289-a53329b0f704 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:02.372021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:02.428430 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9bfe9-3bfa-4081-a289-a53329b0f704 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:02.460174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d69914-5b3b-4439-a5fd-3519b613a061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:02.460764 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d69914-5b3b-4439-a5fd-3519b613a061 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:02.461628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d69914-5b3b-4439-a5fd-3519b613a061 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:02.461900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d69914-5b3b-4439-a5fd-3519b613a061 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:02.462567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d69914-5b3b-4439-a5fd-3519b613a061 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:02.463959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d69914-5b3b-4439-a5fd-3519b613a061 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:02.517129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 596/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Wed Sep 7 16:06:02 2022] POST /identity/v3/domains => generated 267 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:06:04.673583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dce54-6461-4c28-95c1-6b34e7df6194 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:04.674132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dce54-6461-4c28-95c1-6b34e7df6194 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:04.674132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831dce54-6461-4c28-95c1-6b34e7df6194 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:04.675454 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 574/1170] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:06:04.680924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705daa41-4a67-4570-bfe1-79e5429fa9b7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:04.681311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705daa41-4a67-4570-bfe1-79e5429fa9b7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:04.681474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705daa41-4a67-4570-bfe1-79e5429fa9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:04.700663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-705daa41-4a67-4570-bfe1-79e5429fa9b7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:06:04.701850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-705daa41-4a67-4570-bfe1-79e5429fa9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:06:04.765872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 597/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:06:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:06:05.034472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dce54-6461-4c28-95c1-6b34e7df6194 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:05.069427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:05.134792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831dce54-6461-4c28-95c1-6b34e7df6194 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:05.167219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98132443-63e9-4243-8f0c-f3b1cef60b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:05.167856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98132443-63e9-4243-8f0c-f3b1cef60b9b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:05.168131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98132443-63e9-4243-8f0c-f3b1cef60b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:05.168397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98132443-63e9-4243-8f0c-f3b1cef60b9b None admin] PATH_INFO: `/v3/domains/501db3de98f44148b030c600c4f28afe` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:05.178587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98132443-63e9-4243-8f0c-f3b1cef60b9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=501db3de98f44148b030c600c4f28afe)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:05.180555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98132443-63e9-4243-8f0c-f3b1cef60b9b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:05.182123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 575/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Sep 7 16:06:05 2022] GET /identity/v3/domains/501db3de98f44148b030c600c4f28afe => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:05.187992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705daa41-4a67-4570-bfe1-79e5429fa9b7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:05.217011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:05.291788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705daa41-4a67-4570-bfe1-79e5429fa9b7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:05.324260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8a67f-8b5c-480c-bf9c-9371d5a605bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:05.324853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8a67f-8b5c-480c-bf9c-9371d5a605bd None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:05.325135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8a67f-8b5c-480c-bf9c-9371d5a605bd None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:05.325404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8a67f-8b5c-480c-bf9c-9371d5a605bd None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:05.326053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8a67f-8b5c-480c-bf9c-9371d5a605bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:05.327457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8a67f-8b5c-480c-bf9c-9371d5a605bd None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:05.418283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 598/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:06:05 2022] POST /identity/v3/projects => generated 361 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:06:07.594791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399ffd1-2faf-4f69-809b-f78718060495 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:07.595322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399ffd1-2faf-4f69-809b-f78718060495 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:07.595322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399ffd1-2faf-4f69-809b-f78718060495 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:07.596687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 576/1174] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:06:07.602076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3126a-4f14-4919-9e95-2b2911f0a068 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:07.602495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3126a-4f14-4919-9e95-2b2911f0a068 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:07.602683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3126a-4f14-4919-9e95-2b2911f0a068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:07.620535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-f1c3126a-4f14-4919-9e95-2b2911f0a068 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:06:07.621722 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f1c3126a-4f14-4919-9e95-2b2911f0a068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:06:07.686528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 599/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:06:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:06:07.951401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399ffd1-2faf-4f69-809b-f78718060495 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:08.044015 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:08.051510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399ffd1-2faf-4f69-809b-f78718060495 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:08.084889 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976705e-407b-44d8-b053-46c45ee79166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:08.085528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976705e-407b-44d8-b053-46c45ee79166 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:08.085811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976705e-407b-44d8-b053-46c45ee79166 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:08.086116 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976705e-407b-44d8-b053-46c45ee79166 None admin] PATH_INFO: `/v3/domains/501db3de98f44148b030c600c4f28afe` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:08.087376 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976705e-407b-44d8-b053-46c45ee79166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=501db3de98f44148b030c600c4f28afe)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:08.089138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976705e-407b-44d8-b053-46c45ee79166 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:08.091075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 577/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Wed Sep 7 16:06:07 2022] GET /identity/v3/domains/501db3de98f44148b030c600c4f28afe => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:08.096847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3126a-4f14-4919-9e95-2b2911f0a068 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:08.110691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:08.193409 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3126a-4f14-4919-9e95-2b2911f0a068 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:08.226386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f6f5d-63c3-4d2c-9ccc-cb332ad533b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:08.226890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f6f5d-63c3-4d2c-9ccc-cb332ad533b1 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:08.227164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f6f5d-63c3-4d2c-9ccc-cb332ad533b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:08.227438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f6f5d-63c3-4d2c-9ccc-cb332ad533b1 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:08.228122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f6f5d-63c3-4d2c-9ccc-cb332ad533b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:08.229541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f6f5d-63c3-4d2c-9ccc-cb332ad533b1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:08.331442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 600/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:06:08 2022] POST /identity/v3/users => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:06:10.563565 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ebe64-3472-4188-ade4-80d3967505f2 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:10.564142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ebe64-3472-4188-ade4-80d3967505f2 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:10.564142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ebe64-3472-4188-ade4-80d3967505f2 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:10.565463 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 578/1178] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:06:10.571230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9ea1a-dd61-4bf5-902c-653921a93746 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:10.571624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9ea1a-dd61-4bf5-902c-653921a93746 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:10.571783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9ea1a-dd61-4bf5-902c-653921a93746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:10.590064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-32b9ea1a-dd61-4bf5-902c-653921a93746 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:06:10.591235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-32b9ea1a-dd61-4bf5-902c-653921a93746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:06:10.649942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:10.655259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 601/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:06:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:06:10.921510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ebe64-3472-4188-ade4-80d3967505f2 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:11.021672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ebe64-3472-4188-ade4-80d3967505f2 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:11.053880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370b176-ab20-44e6-8413-775c50ef69a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:11.055062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370b176-ab20-44e6-8413-775c50ef69a0 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:11.055336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370b176-ab20-44e6-8413-775c50ef69a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:11.055604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370b176-ab20-44e6-8413-775c50ef69a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:11.065547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370b176-ab20-44e6-8413-775c50ef69a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-5370b176-ab20-44e6-8413-775c50ef69a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:06:11.071113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:06:11.072258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:06:11.073174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 579/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:06:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:06:11.078403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9ea1a-dd61-4bf5-902c-653921a93746 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:11.179484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9ea1a-dd61-4bf5-902c-653921a93746 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:11.212418 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fcef8-b661-4768-9a93-441cc685eda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:11.213019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fcef8-b661-4768-9a93-441cc685eda1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:11.213300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fcef8-b661-4768-9a93-441cc685eda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:11.213576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5fcef8-b661-4768-9a93-441cc685eda1 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:11.214234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fcef8-b661-4768-9a93-441cc685eda1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:06:11.214538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fcef8-b661-4768-9a93-441cc685eda1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:11.215929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5fcef8-b661-4768-9a93-441cc685eda1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:11.223677 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 602/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:06:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:11.229625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370b176-ab20-44e6-8413-775c50ef69a0 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:11.254188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:11.329100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370b176-ab20-44e6-8413-775c50ef69a0 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:11.361676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f00cd-b079-4de7-a0c4-f620a37acd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:11.362284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f00cd-b079-4de7-a0c4-f620a37acd65 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:11.362556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f00cd-b079-4de7-a0c4-f620a37acd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:11.362825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f00cd-b079-4de7-a0c4-f620a37acd65 None admin] PATH_INFO: `/v3/users/e9ca6a6b392d4365994dcfc8a4ec243c` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:11.375776 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f00cd-b079-4de7-a0c4-f620a37acd65 None admin] RBAC: Authorizing `identity:get_user(user_id=e9ca6a6b392d4365994dcfc8a4ec243c)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:11.377214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f00cd-b079-4de7-a0c4-f620a37acd65 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:11.378766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 580/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:06:11 2022] GET /identity/v3/users/e9ca6a6b392d4365994dcfc8a4ec243c => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:11.384534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fcef8-b661-4768-9a93-441cc685eda1 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:11.442193 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:11.481479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5fcef8-b661-4768-9a93-441cc685eda1 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:11.513785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bd1df-d1bb-40e6-a88e-c42f09690b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:11.514368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bd1df-d1bb-40e6-a88e-c42f09690b63 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:11.514644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bd1df-d1bb-40e6-a88e-c42f09690b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:11.514915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bd1df-d1bb-40e6-a88e-c42f09690b63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:11.579040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bd1df-d1bb-40e6-a88e-c42f09690b63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:11.580487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bd1df-d1bb-40e6-a88e-c42f09690b63 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:11.634596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:11.674014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 603/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:06:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:06:11.680290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f00cd-b079-4de7-a0c4-f620a37acd65 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:11.761301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:11.779864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f00cd-b079-4de7-a0c4-f620a37acd65 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:11.813097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af0ce74-9c7e-45b2-8543-b1e8c96b983b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:11.813717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0ce74-9c7e-45b2-8543-b1e8c96b983b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:11.813995 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0ce74-9c7e-45b2-8543-b1e8c96b983b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:11.814269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af0ce74-9c7e-45b2-8543-b1e8c96b983b None admin] PATH_INFO: `/v3/projects/99247bd6afaa4f36b81b30c3b369f7cd` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:11.815369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0ce74-9c7e-45b2-8543-b1e8c96b983b None admin] RBAC: Authorizing `identity:get_project(project_id=99247bd6afaa4f36b81b30c3b369f7cd)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:11.816818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af0ce74-9c7e-45b2-8543-b1e8c96b983b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:11.818439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 581/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:06:11 2022] GET /identity/v3/projects/99247bd6afaa4f36b81b30c3b369f7cd => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:11.824550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bd1df-d1bb-40e6-a88e-c42f09690b63 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:11.919202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bd1df-d1bb-40e6-a88e-c42f09690b63 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:11.951192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965d7ef-e078-4551-a0c5-3c1002151c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:11.951798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965d7ef-e078-4551-a0c5-3c1002151c52 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:11.952073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965d7ef-e078-4551-a0c5-3c1002151c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:11.952342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965d7ef-e078-4551-a0c5-3c1002151c52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:11.953050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965d7ef-e078-4551-a0c5-3c1002151c52 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=99247bd6afaa4f36b81b30c3b369f7cd, user.id=e9ca6a6b392d4365994dcfc8a4ec243c) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:06:11.953438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965d7ef-e078-4551-a0c5-3c1002151c52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:11.954846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965d7ef-e078-4551-a0c5-3c1002151c52 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:11.959965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 604/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:06:11 2022] GET /identity/v3/role_assignments?user.id=e9ca6a6b392d4365994dcfc8a4ec243c&scope.project.id=99247bd6afaa4f36b81b30c3b369f7cd&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:14.064555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83017b-08e7-407f-acf9-8f216a2d269a None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:14.065034 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83017b-08e7-407f-acf9-8f216a2d269a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:14.065123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83017b-08e7-407f-acf9-8f216a2d269a None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:14.066414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 582/1186] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:06:14.073057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b87021-0052-4440-8457-81157e305f77 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:14.073474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b87021-0052-4440-8457-81157e305f77 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:14.073862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b87021-0052-4440-8457-81157e305f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:14.098541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-44b87021-0052-4440-8457-81157e305f77 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:06:14.099780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-44b87021-0052-4440-8457-81157e305f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:06:14.122261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:14.171413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 605/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:06:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:06:14.450824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83017b-08e7-407f-acf9-8f216a2d269a None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:14.548824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83017b-08e7-407f-acf9-8f216a2d269a None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:14.581558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e31a6f-92f1-4136-bb07-7644e6fbc553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:14.582168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e31a6f-92f1-4136-bb07-7644e6fbc553 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:14.582444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e31a6f-92f1-4136-bb07-7644e6fbc553 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:14.582714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e31a6f-92f1-4136-bb07-7644e6fbc553 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:14.592449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e31a6f-92f1-4136-bb07-7644e6fbc553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-e7e31a6f-92f1-4136-bb07-7644e6fbc553 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:06:14.597140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:06:14.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:06:14.599167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 583/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:06:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:06:14.604697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b87021-0052-4440-8457-81157e305f77 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:14.699125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b87021-0052-4440-8457-81157e305f77 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:14.740593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaee1a0-12a0-4ea5-863d-b4953174361a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:14.743130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaee1a0-12a0-4ea5-863d-b4953174361a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:14.743413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaee1a0-12a0-4ea5-863d-b4953174361a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:14.743834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaee1a0-12a0-4ea5-863d-b4953174361a None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:14.744740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaee1a0-12a0-4ea5-863d-b4953174361a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:06:14.745263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaee1a0-12a0-4ea5-863d-b4953174361a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:14.746643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaee1a0-12a0-4ea5-863d-b4953174361a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:14.755871 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 606/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:06:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:14.761804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e31a6f-92f1-4136-bb07-7644e6fbc553 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:14.773963 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:14.858981 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e31a6f-92f1-4136-bb07-7644e6fbc553 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:14.890677 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a6de9-3383-42ed-93bb-6b4dad3cdc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:14.891289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a6de9-3383-42ed-93bb-6b4dad3cdc91 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:14.891564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a6de9-3383-42ed-93bb-6b4dad3cdc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:14.891862 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a6de9-3383-42ed-93bb-6b4dad3cdc91 None admin] PATH_INFO: `/v3/users/e9ca6a6b392d4365994dcfc8a4ec243c` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:14.892959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a6de9-3383-42ed-93bb-6b4dad3cdc91 None admin] RBAC: Authorizing `identity:get_user(user_id=e9ca6a6b392d4365994dcfc8a4ec243c)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:14.894368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a6de9-3383-42ed-93bb-6b4dad3cdc91 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:14.895942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 584/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:06:14 2022] GET /identity/v3/users/e9ca6a6b392d4365994dcfc8a4ec243c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:14.902249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaee1a0-12a0-4ea5-863d-b4953174361a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:14.920626 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:14.999649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaee1a0-12a0-4ea5-863d-b4953174361a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:15.033063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19398db0-e771-4709-abd0-5781fc158f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:15.033679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19398db0-e771-4709-abd0-5781fc158f5a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:15.033954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19398db0-e771-4709-abd0-5781fc158f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:15.034224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19398db0-e771-4709-abd0-5781fc158f5a None admin] PATH_INFO: `/v3/projects/99247bd6afaa4f36b81b30c3b369f7cd` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:15.035310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19398db0-e771-4709-abd0-5781fc158f5a None admin] RBAC: Authorizing `identity:get_project(project_id=99247bd6afaa4f36b81b30c3b369f7cd)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:15.036768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19398db0-e771-4709-abd0-5781fc158f5a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:15.038451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 607/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:06:14 2022] GET /identity/v3/projects/99247bd6afaa4f36b81b30c3b369f7cd => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:15.044620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a6de9-3383-42ed-93bb-6b4dad3cdc91 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:15.120832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:15.143021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a6de9-3383-42ed-93bb-6b4dad3cdc91 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:15.176222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa79682f-bcfe-4f58-b09b-77c7b635e98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:15.176873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79682f-bcfe-4f58-b09b-77c7b635e98f None admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:15.177147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79682f-bcfe-4f58-b09b-77c7b635e98f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:15.177417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa79682f-bcfe-4f58-b09b-77c7b635e98f None admin] PATH_INFO: `/v3/projects/99247bd6afaa4f36b81b30c3b369f7cd/users/e9ca6a6b392d4365994dcfc8a4ec243c/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:15.179555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79682f-bcfe-4f58-b09b-77c7b635e98f None admin] RBAC: Authorizing `identity:create_grant(project_id=99247bd6afaa4f36b81b30c3b369f7cd, user_id=e9ca6a6b392d4365994dcfc8a4ec243c, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:15.181000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa79682f-bcfe-4f58-b09b-77c7b635e98f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:15.274183 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 585/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:06:15 2022] PUT /identity/v3/projects/99247bd6afaa4f36b81b30c3b369f7cd/users/e9ca6a6b392d4365994dcfc8a4ec243c/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:06:17.389080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127248c-4b3d-4f6a-8d82-0514930ddf06 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:17.389586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127248c-4b3d-4f6a-8d82-0514930ddf06 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:17.389698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127248c-4b3d-4f6a-8d82-0514930ddf06 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:17.390953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 608/1193] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:06:17.396097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b31698-752e-4f54-b640-e579ea1d8601 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:17.396463 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b31698-752e-4f54-b640-e579ea1d8601 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:17.396655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b31698-752e-4f54-b640-e579ea1d8601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:17.415666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-21b31698-752e-4f54-b640-e579ea1d8601 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:06:17.416895 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-21b31698-752e-4f54-b640-e579ea1d8601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:06:17.481540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 586/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:06:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:06:17.750694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127248c-4b3d-4f6a-8d82-0514930ddf06 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:17.833480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:17.849654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127248c-4b3d-4f6a-8d82-0514930ddf06 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:17.883080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66df18a-30b3-43af-a597-5fe8862ea0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:17.883697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66df18a-30b3-43af-a597-5fe8862ea0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:17.884003 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66df18a-30b3-43af-a597-5fe8862ea0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:17.884278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66df18a-30b3-43af-a597-5fe8862ea0f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:17.894004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66df18a-30b3-43af-a597-5fe8862ea0f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-d66df18a-30b3-43af-a597-5fe8862ea0f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:06:17.898809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:06:17.899915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:06:17.900847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 609/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:06:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:06:17.905746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b31698-752e-4f54-b640-e579ea1d8601 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:17.968589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:18.002694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b31698-752e-4f54-b640-e579ea1d8601 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:18.035954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9a612-b909-435e-a4bc-965d1d2c5eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:18.036549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9a612-b909-435e-a4bc-965d1d2c5eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:18.036823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9a612-b909-435e-a4bc-965d1d2c5eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:18.037094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9a612-b909-435e-a4bc-965d1d2c5eb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:18.037740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9a612-b909-435e-a4bc-965d1d2c5eb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:06:18.038535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9a612-b909-435e-a4bc-965d1d2c5eb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:18.039899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9a612-b909-435e-a4bc-965d1d2c5eb4 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:18.047598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 587/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:06:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:18.054357 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66df18a-30b3-43af-a597-5fe8862ea0f6 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:18.150108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66df18a-30b3-43af-a597-5fe8862ea0f6 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:18.183235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f69a2-6c10-476c-a8c2-24073b293937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:18.183886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f69a2-6c10-476c-a8c2-24073b293937 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:18.184171 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f69a2-6c10-476c-a8c2-24073b293937 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:18.184449 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f69a2-6c10-476c-a8c2-24073b293937 None admin] PATH_INFO: `/v3/users/e9ca6a6b392d4365994dcfc8a4ec243c` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:18.185567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f69a2-6c10-476c-a8c2-24073b293937 None admin] RBAC: Authorizing `identity:get_user(user_id=e9ca6a6b392d4365994dcfc8a4ec243c)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:18.186993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f69a2-6c10-476c-a8c2-24073b293937 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:18.188574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 610/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Wed Sep 7 16:06:18 2022] GET /identity/v3/users/e9ca6a6b392d4365994dcfc8a4ec243c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:18.194086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9a612-b909-435e-a4bc-965d1d2c5eb4 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:18.289455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9a612-b909-435e-a4bc-965d1d2c5eb4 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:18.315068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:18.322508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853de4de-24c5-47a5-ba2d-7d8a418604e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:18.323991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853de4de-24c5-47a5-ba2d-7d8a418604e1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:18.323991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853de4de-24c5-47a5-ba2d-7d8a418604e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:18.323991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853de4de-24c5-47a5-ba2d-7d8a418604e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:18.388438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853de4de-24c5-47a5-ba2d-7d8a418604e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:18.389871 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853de4de-24c5-47a5-ba2d-7d8a418604e1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:18.484969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 588/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:06:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:06:18.490725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f69a2-6c10-476c-a8c2-24073b293937 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:18.505996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:18.588448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f69a2-6c10-476c-a8c2-24073b293937 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:18.621085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c180c8-22f7-478c-afa0-de67950cd2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:18.621699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c180c8-22f7-478c-afa0-de67950cd2dc None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:18.621981 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c180c8-22f7-478c-afa0-de67950cd2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:18.622259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c180c8-22f7-478c-afa0-de67950cd2dc None admin] PATH_INFO: `/v3/projects/99247bd6afaa4f36b81b30c3b369f7cd` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:18.623361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c180c8-22f7-478c-afa0-de67950cd2dc None admin] RBAC: Authorizing `identity:get_project(project_id=99247bd6afaa4f36b81b30c3b369f7cd)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:18.624819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c180c8-22f7-478c-afa0-de67950cd2dc None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:18.626417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 611/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Wed Sep 7 16:06:18 2022] GET /identity/v3/projects/99247bd6afaa4f36b81b30c3b369f7cd => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:18.632549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853de4de-24c5-47a5-ba2d-7d8a418604e1 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:18.659330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:18.729162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853de4de-24c5-47a5-ba2d-7d8a418604e1 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:18.761676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb5231-fb90-47b0-9860-1779770cccfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:18.762263 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5231-fb90-47b0-9860-1779770cccfe None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:18.762540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5231-fb90-47b0-9860-1779770cccfe None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:18.762810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5231-fb90-47b0-9860-1779770cccfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:18.763521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb5231-fb90-47b0-9860-1779770cccfe None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=99247bd6afaa4f36b81b30c3b369f7cd, user.id=e9ca6a6b392d4365994dcfc8a4ec243c) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:06:18.763851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb5231-fb90-47b0-9860-1779770cccfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:18.765248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb5231-fb90-47b0-9860-1779770cccfe None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:18.770763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 589/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:06:18 2022] GET /identity/v3/role_assignments?user.id=e9ca6a6b392d4365994dcfc8a4ec243c&scope.project.id=99247bd6afaa4f36b81b30c3b369f7cd&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:06:28.616983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=60 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:28.768260 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=54 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:31.578261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204994-abcd-433f-9840-50c19988ee21 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.578867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204994-abcd-433f-9840-50c19988ee21 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.578867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54204994-abcd-433f-9840-50c19988ee21 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.580134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 612/1201] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:07:31.585781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e71be7-c34b-4afd-9f78-8172576c19ce None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.586455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e71be7-c34b-4afd-9f78-8172576c19ce None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.586455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e71be7-c34b-4afd-9f78-8172576c19ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.605376 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-12e71be7-c34b-4afd-9f78-8172576c19ce None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:31.606554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-12e71be7-c34b-4afd-9f78-8172576c19ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:31.672561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 590/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:07:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:07:31.953017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54204994-abcd-433f-9840-50c19988ee21 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:32.054437 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54204994-abcd-433f-9840-50c19988ee21 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:32.088818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809e830-d226-4413-be46-7e5dd9a89a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:32.089446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809e830-d226-4413-be46-7e5dd9a89a5b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:32.089722 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809e830-d226-4413-be46-7e5dd9a89a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:32.089991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1809e830-d226-4413-be46-7e5dd9a89a5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:32.099716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809e830-d226-4413-be46-7e5dd9a89a5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:32.101172 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1809e830-d226-4413-be46-7e5dd9a89a5b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-1809e830-d226-4413-be46-7e5dd9a89a5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:32.115200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:32.116457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:32.117485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:32.117485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:32.117485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:32.117485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 613/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:32.122606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e71be7-c34b-4afd-9f78-8172576c19ce None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:32.219683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e71be7-c34b-4afd-9f78-8172576c19ce None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:32.247351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:32.253280 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c1e9a0-e0e2-49b3-84da-6df123b820e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:32.253873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1e9a0-e0e2-49b3-84da-6df123b820e8 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:32.254145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1e9a0-e0e2-49b3-84da-6df123b820e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:32.254411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c1e9a0-e0e2-49b3-84da-6df123b820e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:32.255037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1e9a0-e0e2-49b3-84da-6df123b820e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:32.255331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1e9a0-e0e2-49b3-84da-6df123b820e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:32.256725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c1e9a0-e0e2-49b3-84da-6df123b820e8 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:32.267468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 591/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:32.273559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809e830-d226-4413-be46-7e5dd9a89a5b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:32.327869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:32.372406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1809e830-d226-4413-be46-7e5dd9a89a5b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:32.404540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03048b0-9db6-4c3d-850c-a718e5e920ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:32.405136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03048b0-9db6-4c3d-850c-a718e5e920ff None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:32.405408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03048b0-9db6-4c3d-850c-a718e5e920ff None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:32.405676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03048b0-9db6-4c3d-850c-a718e5e920ff None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:32.406316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03048b0-9db6-4c3d-850c-a718e5e920ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:32.407803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03048b0-9db6-4c3d-850c-a718e5e920ff None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:32.572414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 614/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Wed Sep 7 16:07:32 2022] POST /identity/v3/users => generated 251 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:34.735555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aab7f4-1144-4f4b-9674-9cce34c169be None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:34.736121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aab7f4-1144-4f4b-9674-9cce34c169be None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:34.736121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aab7f4-1144-4f4b-9674-9cce34c169be None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:34.737505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 592/1206] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:07:34.742799 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb97593-6ab4-49d0-a58f-2c732644fc7d None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:34.743196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb97593-6ab4-49d0-a58f-2c732644fc7d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:34.743399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb97593-6ab4-49d0-a58f-2c732644fc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:34.761239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-0bb97593-6ab4-49d0-a58f-2c732644fc7d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:34.762420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0bb97593-6ab4-49d0-a58f-2c732644fc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:34.828229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 615/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:07:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:07:35.094816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aab7f4-1144-4f4b-9674-9cce34c169be None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:35.193520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aab7f4-1144-4f4b-9674-9cce34c169be None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:35.226406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6243a8-cf6b-42fe-a615-345616f75961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:35.227018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6243a8-cf6b-42fe-a615-345616f75961 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:35.227293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6243a8-cf6b-42fe-a615-345616f75961 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:35.227560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6243a8-cf6b-42fe-a615-345616f75961 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:35.237886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6243a8-cf6b-42fe-a615-345616f75961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-ef6243a8-cf6b-42fe-a615-345616f75961 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:07:35.242537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:35.246588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:35.247640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 593/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:07:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:35.249582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb97593-6ab4-49d0-a58f-2c732644fc7d None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:35.292697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:35.349273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb97593-6ab4-49d0-a58f-2c732644fc7d None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:35.382363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e1aa1-1db2-4efd-beeb-deae282fc1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:35.382962 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e1aa1-1db2-4efd-beeb-deae282fc1e1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:35.383237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e1aa1-1db2-4efd-beeb-deae282fc1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:35.383506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e1aa1-1db2-4efd-beeb-deae282fc1e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:35.384219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e1aa1-1db2-4efd-beeb-deae282fc1e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:35.384483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e1aa1-1db2-4efd-beeb-deae282fc1e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:35.385826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e1aa1-1db2-4efd-beeb-deae282fc1e1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:35.393332 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 616/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:07:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:35.399371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6243a8-cf6b-42fe-a615-345616f75961 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:35.449012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:35.495481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6243a8-cf6b-42fe-a615-345616f75961 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:35.530694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e194a81-6ce3-435d-9d0a-8afcb018f8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:35.531425 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e194a81-6ce3-435d-9d0a-8afcb018f8bb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:35.531704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e194a81-6ce3-435d-9d0a-8afcb018f8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:35.532003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e194a81-6ce3-435d-9d0a-8afcb018f8bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:35.542009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e194a81-6ce3-435d-9d0a-8afcb018f8bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:35.544790 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e194a81-6ce3-435d-9d0a-8afcb018f8bb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-3e194a81-6ce3-435d-9d0a-8afcb018f8bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:35.559327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:35.560569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:35.561568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:35.561568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:35.561568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:35.561568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 594/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:35.567070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e1aa1-1db2-4efd-beeb-deae282fc1e1 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:35.664661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:35.667526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e1aa1-1db2-4efd-beeb-deae282fc1e1 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:35.700849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84adb14-3aae-4944-9968-a175d3369084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:35.701449 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84adb14-3aae-4944-9968-a175d3369084 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:35.701725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84adb14-3aae-4944-9968-a175d3369084 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:35.701996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84adb14-3aae-4944-9968-a175d3369084 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:35.702623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84adb14-3aae-4944-9968-a175d3369084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:35.702921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84adb14-3aae-4944-9968-a175d3369084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:35.704324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84adb14-3aae-4944-9968-a175d3369084 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:35.715003 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 617/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:35.721420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e194a81-6ce3-435d-9d0a-8afcb018f8bb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:35.822002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e194a81-6ce3-435d-9d0a-8afcb018f8bb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:35.823416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:35.855849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5c7b0-88b4-4855-a5af-eb7c6257b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:35.856465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5c7b0-88b4-4855-a5af-eb7c6257b1af None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:35.856742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5c7b0-88b4-4855-a5af-eb7c6257b1af None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:35.857012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5c7b0-88b4-4855-a5af-eb7c6257b1af None admin] PATH_INFO: `/v3/users/placement` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:35.864309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5c7b0-88b4-4855-a5af-eb7c6257b1af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:35.870237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5c7b0-88b4-4855-a5af-eb7c6257b1af None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-d7e5c7b0-88b4-4855-a5af-eb7c6257b1af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:07:35.877715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:35.878830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:35.879818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:35.879818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:35.879818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:07:35.879818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:07:35.879818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:07:35.879818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:07:35.879818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:07:35.879818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:07:35.879818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:35.879818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:35.879818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 595/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:35.884742 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84adb14-3aae-4944-9968-a175d3369084 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:35.981835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84adb14-3aae-4944-9968-a175d3369084 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:36.020325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a6953-f9fe-4c69-b151-3a69cf9c9c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:36.020930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a6953-f9fe-4c69-b151-3a69cf9c9c6a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:36.021208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a6953-f9fe-4c69-b151-3a69cf9c9c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:36.021480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a6953-f9fe-4c69-b151-3a69cf9c9c6a None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:36.022198 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a6953-f9fe-4c69-b151-3a69cf9c9c6a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:36.022499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a6953-f9fe-4c69-b151-3a69cf9c9c6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:36.023958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a6953-f9fe-4c69-b151-3a69cf9c9c6a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:36.041503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 618/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 7 16:07:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:36.047486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5c7b0-88b4-4855-a5af-eb7c6257b1af None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:36.147717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5c7b0-88b4-4855-a5af-eb7c6257b1af None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:36.181021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222c392-527e-4111-9bf5-4589c55db407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:36.181617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c392-527e-4111-9bf5-4589c55db407 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:36.181894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c392-527e-4111-9bf5-4589c55db407 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:36.182168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222c392-527e-4111-9bf5-4589c55db407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:36.211557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:36.248154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222c392-527e-4111-9bf5-4589c55db407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:36.249600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222c392-527e-4111-9bf5-4589c55db407 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:36.346515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 596/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:07:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:07:36.352401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a6953-f9fe-4c69-b151-3a69cf9c9c6a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:36.371611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:36.451056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a6953-f9fe-4c69-b151-3a69cf9c9c6a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:36.484259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565c6af-b86c-42f8-ae16-0c29cc94613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:36.484880 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c6af-b86c-42f8-ae16-0c29cc94613b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:36.485155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c6af-b86c-42f8-ae16-0c29cc94613b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:36.485424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565c6af-b86c-42f8-ae16-0c29cc94613b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:36.495969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565c6af-b86c-42f8-ae16-0c29cc94613b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:36.497391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565c6af-b86c-42f8-ae16-0c29cc94613b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-2565c6af-b86c-42f8-ae16-0c29cc94613b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:36.511446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:36.517328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:36.518248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:36.518248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:36.518248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:36.518248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 619/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:36.518486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222c392-527e-4111-9bf5-4589c55db407 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:36.577927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:36.615371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222c392-527e-4111-9bf5-4589c55db407 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:36.650933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b66ec4-f0cd-48a1-874a-22732473790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:36.651546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b66ec4-f0cd-48a1-874a-22732473790d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:36.651858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b66ec4-f0cd-48a1-874a-22732473790d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:36.652132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b66ec4-f0cd-48a1-874a-22732473790d None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:36.652773 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b66ec4-f0cd-48a1-874a-22732473790d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:36.653076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b66ec4-f0cd-48a1-874a-22732473790d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:36.654479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b66ec4-f0cd-48a1-874a-22732473790d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:36.666744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 597/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:36.672456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565c6af-b86c-42f8-ae16-0c29cc94613b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:36.741237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:36.770737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565c6af-b86c-42f8-ae16-0c29cc94613b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:36.803229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f60147-8cc8-46aa-8cbf-d5add77ad501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:36.803898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f60147-8cc8-46aa-8cbf-d5add77ad501 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:36.804174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f60147-8cc8-46aa-8cbf-d5add77ad501 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:36.804443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f60147-8cc8-46aa-8cbf-d5add77ad501 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:36.810800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f60147-8cc8-46aa-8cbf-d5add77ad501 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:36.812272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f60147-8cc8-46aa-8cbf-d5add77ad501 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-13f60147-8cc8-46aa-8cbf-d5add77ad501 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:07:36.823210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:36.824380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:36.825503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:07:36.825503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:07:36.825503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:07:36.825503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:07:36.825503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:07:36.825503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:07:36.825503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:36.825503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:36.825503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 620/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:07:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:36.830726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b66ec4-f0cd-48a1-874a-22732473790d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:36.929580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b66ec4-f0cd-48a1-874a-22732473790d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:36.951319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:36.963277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855a259-24e2-40ad-9208-86daa65a2423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:36.963915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855a259-24e2-40ad-9208-86daa65a2423 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:36.964198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855a259-24e2-40ad-9208-86daa65a2423 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:36.964472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855a259-24e2-40ad-9208-86daa65a2423 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:36.965186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855a259-24e2-40ad-9208-86daa65a2423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:36.965485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855a259-24e2-40ad-9208-86daa65a2423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:36.966918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855a259-24e2-40ad-9208-86daa65a2423 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:36.978917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 598/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:07:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:36.985283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f60147-8cc8-46aa-8cbf-d5add77ad501 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:37.086391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f60147-8cc8-46aa-8cbf-d5add77ad501 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:37.116687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:37.122853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c459814-7e1d-43e6-b957-bf4d8582d24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:37.123458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c459814-7e1d-43e6-b957-bf4d8582d24f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:37.123734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c459814-7e1d-43e6-b957-bf4d8582d24f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:37.124045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c459814-7e1d-43e6-b957-bf4d8582d24f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:37.127324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c459814-7e1d-43e6-b957-bf4d8582d24f None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=5fdfb0bf09db406ab5c09fbef887cbab) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:37.127324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c459814-7e1d-43e6-b957-bf4d8582d24f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:37.127324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c459814-7e1d-43e6-b957-bf4d8582d24f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:37.132932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 621/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:07:36 2022] GET /identity/v3/role_assignments?user.id=5fdfb0bf09db406ab5c09fbef887cbab&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:39.266000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f63a71-c194-41f2-b5f8-4682417c6a84 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:39.266494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f63a71-c194-41f2-b5f8-4682417c6a84 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:39.266494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f63a71-c194-41f2-b5f8-4682417c6a84 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:39.268385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 599/1220] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:07:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:07:39.273664 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847cf82-a5f2-4f9a-90df-1421b4f266a6 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:39.274062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847cf82-a5f2-4f9a-90df-1421b4f266a6 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:39.274283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6847cf82-a5f2-4f9a-90df-1421b4f266a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:39.293231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6847cf82-a5f2-4f9a-90df-1421b4f266a6 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:39.294407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6847cf82-a5f2-4f9a-90df-1421b4f266a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:39.360252 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 622/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:07:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:07:39.626551 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f63a71-c194-41f2-b5f8-4682417c6a84 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:39.728628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f63a71-c194-41f2-b5f8-4682417c6a84 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:39.761443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf0742-e9c4-4348-873f-64cca052f072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:39.762059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf0742-e9c4-4348-873f-64cca052f072 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:39.762338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf0742-e9c4-4348-873f-64cca052f072 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:39.762609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cf0742-e9c4-4348-873f-64cca052f072 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:39.772766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cf0742-e9c4-4348-873f-64cca052f072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-07cf0742-e9c4-4348-873f-64cca052f072 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:07:39.777465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:39.778583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:39.779431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 600/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:07:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:07:39.784705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847cf82-a5f2-4f9a-90df-1421b4f266a6 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:39.880195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6847cf82-a5f2-4f9a-90df-1421b4f266a6 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:39.904144 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:39.914235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d7606-d070-4398-b12a-4b6135e152d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:39.914835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d7606-d070-4398-b12a-4b6135e152d9 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:39.919338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d7606-d070-4398-b12a-4b6135e152d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:39.919338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d7606-d070-4398-b12a-4b6135e152d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:39.919338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d7606-d070-4398-b12a-4b6135e152d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:39.919338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d7606-d070-4398-b12a-4b6135e152d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:39.919338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2d7606-d070-4398-b12a-4b6135e152d9 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:39.926358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 623/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:07:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:39.932311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf0742-e9c4-4348-873f-64cca052f072 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:39.975223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:40.030742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cf0742-e9c4-4348-873f-64cca052f072 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.063745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f52b31-2d5f-4be6-8480-13923dbe570d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.065001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f52b31-2d5f-4be6-8480-13923dbe570d None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.065541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f52b31-2d5f-4be6-8480-13923dbe570d None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.065812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f52b31-2d5f-4be6-8480-13923dbe570d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.076184 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f52b31-2d5f-4be6-8480-13923dbe570d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.077610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f52b31-2d5f-4be6-8480-13923dbe570d None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-d5f52b31-2d5f-4be6-8480-13923dbe570d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:40.091230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:40.092369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:40.093424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:40.093424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:40.093424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:40.093424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 601/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:40.100330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d7606-d070-4398-b12a-4b6135e152d9 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.200948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d7606-d070-4398-b12a-4b6135e152d9 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.233573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591203ff-b382-423d-b459-83d0b1fd51ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.234174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591203ff-b382-423d-b459-83d0b1fd51ea None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.234451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591203ff-b382-423d-b459-83d0b1fd51ea None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.234724 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591203ff-b382-423d-b459-83d0b1fd51ea None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.235352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591203ff-b382-423d-b459-83d0b1fd51ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:40.235652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591203ff-b382-423d-b459-83d0b1fd51ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.237642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591203ff-b382-423d-b459-83d0b1fd51ea None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.249026 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 624/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:40.254869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f52b31-2d5f-4be6-8480-13923dbe570d None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.346830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:40.353450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f52b31-2d5f-4be6-8480-13923dbe570d None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.385916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3245a-4f35-497a-b774-28d2ec4bcfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.386533 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3245a-4f35-497a-b774-28d2ec4bcfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.386809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3245a-4f35-497a-b774-28d2ec4bcfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.387079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3245a-4f35-497a-b774-28d2ec4bcfe9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.393818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3245a-4f35-497a-b774-28d2ec4bcfe9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.396618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3245a-4f35-497a-b774-28d2ec4bcfe9 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-07d3245a-4f35-497a-b774-28d2ec4bcfe9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:07:40.406728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:40.407668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:40.409232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:40.409232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:40.409232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:07:40.409232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:07:40.409232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:07:40.409232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:07:40.409232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:07:40.409232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:07:40.409232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:40.409232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:40.409232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 602/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:40.414250 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591203ff-b382-423d-b459-83d0b1fd51ea None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.461050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:40.511951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591203ff-b382-423d-b459-83d0b1fd51ea None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.545087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8a935-ecfe-4c0b-9dea-ff6655e472d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.545684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8a935-ecfe-4c0b-9dea-ff6655e472d9 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.545961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8a935-ecfe-4c0b-9dea-ff6655e472d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.546228 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f8a935-ecfe-4c0b-9dea-ff6655e472d9 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.546931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8a935-ecfe-4c0b-9dea-ff6655e472d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:40.547231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8a935-ecfe-4c0b-9dea-ff6655e472d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.548923 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f8a935-ecfe-4c0b-9dea-ff6655e472d9 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.565392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 625/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 7 16:07:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:40.571231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3245a-4f35-497a-b774-28d2ec4bcfe9 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.669797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3245a-4f35-497a-b774-28d2ec4bcfe9 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.702197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e6792-5208-47bb-90fe-48da3daba9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.702840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e6792-5208-47bb-90fe-48da3daba9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.703112 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e6792-5208-47bb-90fe-48da3daba9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.703378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e6792-5208-47bb-90fe-48da3daba9d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.715377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e6792-5208-47bb-90fe-48da3daba9d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.716857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92e6792-5208-47bb-90fe-48da3daba9d2 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-c92e6792-5208-47bb-90fe-48da3daba9d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:40.731139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:40.732235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:40.733351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:40.733351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:40.733351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:40.733351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 603/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:40.739790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8a935-ecfe-4c0b-9dea-ff6655e472d9 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.849016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f8a935-ecfe-4c0b-9dea-ff6655e472d9 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.851358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:40.883899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd322361-26e6-4613-870c-5ba08f7bc461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.884499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd322361-26e6-4613-870c-5ba08f7bc461 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.884776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd322361-26e6-4613-870c-5ba08f7bc461 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.885045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd322361-26e6-4613-870c-5ba08f7bc461 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.885668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd322361-26e6-4613-870c-5ba08f7bc461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:40.885964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd322361-26e6-4613-870c-5ba08f7bc461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.887318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd322361-26e6-4613-870c-5ba08f7bc461 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.898951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 626/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:40.904823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e6792-5208-47bb-90fe-48da3daba9d2 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.914313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:41.001124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92e6792-5208-47bb-90fe-48da3daba9d2 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:41.036111 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfc238-1746-4c5c-bccb-d9e6eb52dbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:41.036731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfc238-1746-4c5c-bccb-d9e6eb52dbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:41.037002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfc238-1746-4c5c-bccb-d9e6eb52dbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:41.037267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfc238-1746-4c5c-bccb-d9e6eb52dbf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:41.043759 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfc238-1746-4c5c-bccb-d9e6eb52dbf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:41.045264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfc238-1746-4c5c-bccb-d9e6eb52dbf7 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-efdfc238-1746-4c5c-bccb-d9e6eb52dbf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:07:41.056310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:41.057372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:41.058469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:07:41.058469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:07:41.058469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:07:41.058469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:07:41.058469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:07:41.058469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:07:41.058469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:41.058469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:41.058469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 604/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:07:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:07:41.063283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd322361-26e6-4613-870c-5ba08f7bc461 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:41.170586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd322361-26e6-4613-870c-5ba08f7bc461 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:41.204304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5349582-0f22-4d68-831a-6da0aa2dc041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:41.205471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5349582-0f22-4d68-831a-6da0aa2dc041 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:41.205759 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5349582-0f22-4d68-831a-6da0aa2dc041 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:41.206032 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5349582-0f22-4d68-831a-6da0aa2dc041 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:41.206756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5349582-0f22-4d68-831a-6da0aa2dc041 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:41.207059 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5349582-0f22-4d68-831a-6da0aa2dc041 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:41.208614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5349582-0f22-4d68-831a-6da0aa2dc041 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:41.220365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 627/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:07:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:41.226462 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfc238-1746-4c5c-bccb-d9e6eb52dbf7 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:41.290755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:41.326163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfc238-1746-4c5c-bccb-d9e6eb52dbf7 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:41.359480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e67465-e912-4d2d-8f53-4fa4c23ea36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:41.360163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e67465-e912-4d2d-8f53-4fa4c23ea36e None admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:41.360440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e67465-e912-4d2d-8f53-4fa4c23ea36e None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:41.360763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e67465-e912-4d2d-8f53-4fa4c23ea36e None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/5fdfb0bf09db406ab5c09fbef887cbab/roles/d9477edba6304c829c6348bc4e6e6c1d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:41.375234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e67465-e912-4d2d-8f53-4fa4c23ea36e None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=5fdfb0bf09db406ab5c09fbef887cbab, role_id=d9477edba6304c829c6348bc4e6e6c1d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:41.377261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e67465-e912-4d2d-8f53-4fa4c23ea36e None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:41.426679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 605/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:07:41 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/5fdfb0bf09db406ab5c09fbef887cbab/roles/d9477edba6304c829c6348bc4e6e6c1d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:43.557514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7068683-cfda-4b11-aa22-33b2ed2d0699 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:43.558076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7068683-cfda-4b11-aa22-33b2ed2d0699 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:43.558076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7068683-cfda-4b11-aa22-33b2ed2d0699 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:43.559393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 628/1233] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:07:43.564509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563bc5b-048d-432f-b9e9-cdb5d43e5d0c None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:43.564992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563bc5b-048d-432f-b9e9-cdb5d43e5d0c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:43.565241 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563bc5b-048d-432f-b9e9-cdb5d43e5d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:43.583615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-f563bc5b-048d-432f-b9e9-cdb5d43e5d0c None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:43.584839 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f563bc5b-048d-432f-b9e9-cdb5d43e5d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:43.653366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 606/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:07:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:07:43.924256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7068683-cfda-4b11-aa22-33b2ed2d0699 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:43.950318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:44.026509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7068683-cfda-4b11-aa22-33b2ed2d0699 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:44.060756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d3549-ff76-4a89-9997-18c25b532aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:44.061382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d3549-ff76-4a89-9997-18c25b532aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:44.061663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d3549-ff76-4a89-9997-18c25b532aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:44.061936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0d3549-ff76-4a89-9997-18c25b532aa5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:44.072384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0d3549-ff76-4a89-9997-18c25b532aa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-df0d3549-ff76-4a89-9997-18c25b532aa5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:07:44.077235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:44.078335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:44.080069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 629/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Wed Sep 7 16:07:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:44.084890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563bc5b-048d-432f-b9e9-cdb5d43e5d0c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:44.132418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:44.182798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563bc5b-048d-432f-b9e9-cdb5d43e5d0c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:44.216005 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb29ad-1706-42b3-a98a-da9bd6f59b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:44.216613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb29ad-1706-42b3-a98a-da9bd6f59b95 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:44.216891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb29ad-1706-42b3-a98a-da9bd6f59b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:44.217181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb29ad-1706-42b3-a98a-da9bd6f59b95 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:44.217970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb29ad-1706-42b3-a98a-da9bd6f59b95 None admin] RBAC: Adding query filter params (name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:44.218283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb29ad-1706-42b3-a98a-da9bd6f59b95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:44.219658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb29ad-1706-42b3-a98a-da9bd6f59b95 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:44.227358 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 607/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Wed Sep 7 16:07:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:44.233235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d3549-ff76-4a89-9997-18c25b532aa5 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:44.317391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:44.332773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0d3549-ff76-4a89-9997-18c25b532aa5 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:44.365664 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e0ceb-839f-4926-8d08-70edcf5e1424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:44.366281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e0ceb-839f-4926-8d08-70edcf5e1424 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:44.366556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e0ceb-839f-4926-8d08-70edcf5e1424 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:44.366827 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e0ceb-839f-4926-8d08-70edcf5e1424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:44.377166 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e0ceb-839f-4926-8d08-70edcf5e1424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:44.380792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e0ceb-839f-4926-8d08-70edcf5e1424 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-bb3e0ceb-839f-4926-8d08-70edcf5e1424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:44.392840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:44.393872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:44.394832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:44.394832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:44.394832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:44.394832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 630/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:44.399882 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb29ad-1706-42b3-a98a-da9bd6f59b95 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:44.498693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb29ad-1706-42b3-a98a-da9bd6f59b95 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:44.508178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:44.532003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f8bc3-bd1f-47c6-85d4-4002e4c89027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:44.532605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f8bc3-bd1f-47c6-85d4-4002e4c89027 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:44.532885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f8bc3-bd1f-47c6-85d4-4002e4c89027 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:44.533157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f8bc3-bd1f-47c6-85d4-4002e4c89027 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:44.533794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f8bc3-bd1f-47c6-85d4-4002e4c89027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:44.534098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f8bc3-bd1f-47c6-85d4-4002e4c89027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:44.535456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f8bc3-bd1f-47c6-85d4-4002e4c89027 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:44.546979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 608/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:44.553601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e0ceb-839f-4926-8d08-70edcf5e1424 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:44.652008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e0ceb-839f-4926-8d08-70edcf5e1424 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:44.685486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e0e1f-de29-4472-b648-7f42bd07ced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:44.686109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e0e1f-de29-4472-b648-7f42bd07ced7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:44.686391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e0e1f-de29-4472-b648-7f42bd07ced7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:44.686668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e0e1f-de29-4472-b648-7f42bd07ced7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:44.689455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:44.694286 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e0e1f-de29-4472-b648-7f42bd07ced7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:44.695724 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e0e1f-de29-4472-b648-7f42bd07ced7 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-6a7e0e1f-de29-4472-b648-7f42bd07ced7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:07:44.706550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:44.707697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:44.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:44.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:44.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:07:44.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:07:44.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:07:44.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:07:44.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:07:44.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:07:44.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:44.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:44.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 631/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:44.713660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f8bc3-bd1f-47c6-85d4-4002e4c89027 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:44.812118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f8bc3-bd1f-47c6-85d4-4002e4c89027 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:44.844345 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc10ba7-e0f3-49b1-ae47-46811f42f7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:44.845530 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc10ba7-e0f3-49b1-ae47-46811f42f7cc None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:44.845808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc10ba7-e0f3-49b1-ae47-46811f42f7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:44.846075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc10ba7-e0f3-49b1-ae47-46811f42f7cc None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:44.846781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc10ba7-e0f3-49b1-ae47-46811f42f7cc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:44.847078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc10ba7-e0f3-49b1-ae47-46811f42f7cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:44.848530 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc10ba7-e0f3-49b1-ae47-46811f42f7cc None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:44.865535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 609/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 7 16:07:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:44.873283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e0e1f-de29-4472-b648-7f42bd07ced7 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:44.969494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e0e1f-de29-4472-b648-7f42bd07ced7 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:45.002848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7eb9b-132e-4061-8d15-a820aa99ccc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:45.003448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7eb9b-132e-4061-8d15-a820aa99ccc5 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:45.003727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7eb9b-132e-4061-8d15-a820aa99ccc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:45.004028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7eb9b-132e-4061-8d15-a820aa99ccc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:45.066380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:45.072786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7eb9b-132e-4061-8d15-a820aa99ccc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:45.074307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7eb9b-132e-4061-8d15-a820aa99ccc5 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:45.178950 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 632/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:07:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 16:07:45.185081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc10ba7-e0f3-49b1-ae47-46811f42f7cc None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:45.212258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:45.280867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc10ba7-e0f3-49b1-ae47-46811f42f7cc None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:45.315353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce02d26-ed9a-47c4-8a37-5757f0c1d68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:45.316001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce02d26-ed9a-47c4-8a37-5757f0c1d68b None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:45.316278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce02d26-ed9a-47c4-8a37-5757f0c1d68b None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:45.316592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce02d26-ed9a-47c4-8a37-5757f0c1d68b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:45.326257 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce02d26-ed9a-47c4-8a37-5757f0c1d68b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:45.327680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce02d26-ed9a-47c4-8a37-5757f0c1d68b None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-1ce02d26-ed9a-47c4-8a37-5757f0c1d68b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:45.342253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:45.343347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:45.348094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:45.348094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:45.348094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:45.348094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 610/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:45.350005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7eb9b-132e-4061-8d15-a820aa99ccc5 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:45.441629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:45.447529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7eb9b-132e-4061-8d15-a820aa99ccc5 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:45.480758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52f116-55a7-4d74-b98b-03b716b9cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:45.481394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52f116-55a7-4d74-b98b-03b716b9cf19 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:45.481670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52f116-55a7-4d74-b98b-03b716b9cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:45.481940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52f116-55a7-4d74-b98b-03b716b9cf19 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:45.482876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52f116-55a7-4d74-b98b-03b716b9cf19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:45.483386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52f116-55a7-4d74-b98b-03b716b9cf19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:45.485934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52f116-55a7-4d74-b98b-03b716b9cf19 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:45.500666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 633/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:45.507159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce02d26-ed9a-47c4-8a37-5757f0c1d68b None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:45.586009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:45.606354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce02d26-ed9a-47c4-8a37-5757f0c1d68b None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:45.639226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed57f4-1cb5-462f-b5ae-be8e5aed63f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:45.639892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed57f4-1cb5-462f-b5ae-be8e5aed63f1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:45.640170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed57f4-1cb5-462f-b5ae-be8e5aed63f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:45.640441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed57f4-1cb5-462f-b5ae-be8e5aed63f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:45.647672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed57f4-1cb5-462f-b5ae-be8e5aed63f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:45.649126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed57f4-1cb5-462f-b5ae-be8e5aed63f1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-a1ed57f4-1cb5-462f-b5ae-be8e5aed63f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:07:45.660078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:45.661109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:45.662053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:07:45.662053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:07:45.662053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:07:45.662053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:07:45.662053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:07:45.662053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:07:45.662053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:45.662053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:45.662053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 611/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:07:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:45.667209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52f116-55a7-4d74-b98b-03b716b9cf19 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:45.767073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52f116-55a7-4d74-b98b-03b716b9cf19 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:45.804750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf4798-ad14-4467-99f0-8bb81a19844f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:45.805362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf4798-ad14-4467-99f0-8bb81a19844f None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:45.805645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf4798-ad14-4467-99f0-8bb81a19844f None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:45.805917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf4798-ad14-4467-99f0-8bb81a19844f None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:45.806646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf4798-ad14-4467-99f0-8bb81a19844f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:45.806953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf4798-ad14-4467-99f0-8bb81a19844f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:45.809468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bf4798-ad14-4467-99f0-8bb81a19844f None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:45.822924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 634/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:07:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:45.829324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed57f4-1cb5-462f-b5ae-be8e5aed63f1 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:45.930533 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed57f4-1cb5-462f-b5ae-be8e5aed63f1 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:45.963965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d7570-362f-43b5-a70b-a18c4dd3c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:45.964562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d7570-362f-43b5-a70b-a18c4dd3c778 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:45.964843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d7570-362f-43b5-a70b-a18c4dd3c778 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:45.965117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d7570-362f-43b5-a70b-a18c4dd3c778 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:45.965834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d7570-362f-43b5-a70b-a18c4dd3c778 None admin] RBAC: Adding query filter params (role.id=d9477edba6304c829c6348bc4e6e6c1d, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=5fdfb0bf09db406ab5c09fbef887cbab) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:45.966143 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d7570-362f-43b5-a70b-a18c4dd3c778 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:45.967565 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d7570-362f-43b5-a70b-a18c4dd3c778 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:45.973256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 612/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:07:45 2022] GET /identity/v3/role_assignments?user.id=5fdfb0bf09db406ab5c09fbef887cbab&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=d9477edba6304c829c6348bc4e6e6c1d => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:48.209120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6fe44-dab8-41d4-a219-d7f3c6b043b4 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:48.209637 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6fe44-dab8-41d4-a219-d7f3c6b043b4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:48.209750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6fe44-dab8-41d4-a219-d7f3c6b043b4 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:48.211093 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 635/1247] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:07:48.216146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1595a5-fe02-4d4c-899e-24f22ae9c51b None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:48.216551 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1595a5-fe02-4d4c-899e-24f22ae9c51b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:48.216695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1595a5-fe02-4d4c-899e-24f22ae9c51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:48.223641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:48.235810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-3d1595a5-fe02-4d4c-899e-24f22ae9c51b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:48.237026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3d1595a5-fe02-4d4c-899e-24f22ae9c51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:48.302149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 613/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:07:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:07:48.569919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6fe44-dab8-41d4-a219-d7f3c6b043b4 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:48.588319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:48.671141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6fe44-dab8-41d4-a219-d7f3c6b043b4 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:48.704663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa784c5-cac4-4cc7-bda9-d85dff602466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:48.705286 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa784c5-cac4-4cc7-bda9-d85dff602466 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:48.705559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa784c5-cac4-4cc7-bda9-d85dff602466 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:48.705822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa784c5-cac4-4cc7-bda9-d85dff602466 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:48.715471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa784c5-cac4-4cc7-bda9-d85dff602466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-0fa784c5-cac4-4cc7-bda9-d85dff602466 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:07:48.720270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:48.721216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:48.722129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 636/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:07:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:48.727455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1595a5-fe02-4d4c-899e-24f22ae9c51b None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:48.826312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1595a5-fe02-4d4c-899e-24f22ae9c51b None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:48.854644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:48.859476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ddca4-c115-4b79-9fe3-c4cc879e9424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:48.860103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ddca4-c115-4b79-9fe3-c4cc879e9424 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:48.860890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ddca4-c115-4b79-9fe3-c4cc879e9424 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:48.861165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ddca4-c115-4b79-9fe3-c4cc879e9424 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:48.861826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ddca4-c115-4b79-9fe3-c4cc879e9424 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:48.862127 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ddca4-c115-4b79-9fe3-c4cc879e9424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:48.863493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ddca4-c115-4b79-9fe3-c4cc879e9424 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:48.871292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 614/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:07:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:48.877699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa784c5-cac4-4cc7-bda9-d85dff602466 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:48.953775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:48.976476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa784c5-cac4-4cc7-bda9-d85dff602466 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:49.010495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea181c-62d3-482f-801e-0f3cdc949757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:49.011127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea181c-62d3-482f-801e-0f3cdc949757 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:49.011406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea181c-62d3-482f-801e-0f3cdc949757 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:49.011684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea181c-62d3-482f-801e-0f3cdc949757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:49.021478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea181c-62d3-482f-801e-0f3cdc949757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:49.022951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea181c-62d3-482f-801e-0f3cdc949757 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-73ea181c-62d3-482f-801e-0f3cdc949757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:49.038356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:49.039551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:49.040551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:49.040551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:49.040551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:49.040551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 637/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 16:07:49.045263 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ddca4-c115-4b79-9fe3-c4cc879e9424 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:49.143964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ddca4-c115-4b79-9fe3-c4cc879e9424 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:49.177507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70bf88-657a-425d-a611-9f8508c784ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:49.178100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70bf88-657a-425d-a611-9f8508c784ac None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:49.178381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70bf88-657a-425d-a611-9f8508c784ac None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:49.178651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70bf88-657a-425d-a611-9f8508c784ac None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:49.179287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70bf88-657a-425d-a611-9f8508c784ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:49.179589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70bf88-657a-425d-a611-9f8508c784ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:49.180969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70bf88-657a-425d-a611-9f8508c784ac None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:49.192386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 615/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:49.198113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea181c-62d3-482f-801e-0f3cdc949757 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:49.297534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea181c-62d3-482f-801e-0f3cdc949757 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:49.323926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:49.331558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ed1d1-b8c2-480f-98b2-fe89242f6855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:49.332203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ed1d1-b8c2-480f-98b2-fe89242f6855 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:49.332482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ed1d1-b8c2-480f-98b2-fe89242f6855 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:49.332754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ed1d1-b8c2-480f-98b2-fe89242f6855 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:49.340392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ed1d1-b8c2-480f-98b2-fe89242f6855 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:49.341869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ed1d1-b8c2-480f-98b2-fe89242f6855 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-565ed1d1-b8c2-480f-98b2-fe89242f6855 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:07:49.353555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:49.354682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:49.355662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:49.355662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:49.355662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:07:49.355662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:07:49.355662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:07:49.355662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:07:49.355662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:07:49.355662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:07:49.355662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:49.355662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:49.355662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 638/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:49.361173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70bf88-657a-425d-a611-9f8508c784ac None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:49.415335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:49.458184 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70bf88-657a-425d-a611-9f8508c784ac None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:49.491312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a56c7-4a89-4db0-b71f-4f938a127dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:49.491940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a56c7-4a89-4db0-b71f-4f938a127dec None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:49.492224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a56c7-4a89-4db0-b71f-4f938a127dec None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:49.492504 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a56c7-4a89-4db0-b71f-4f938a127dec None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:49.493222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a56c7-4a89-4db0-b71f-4f938a127dec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:49.493531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a56c7-4a89-4db0-b71f-4f938a127dec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:49.494957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4a56c7-4a89-4db0-b71f-4f938a127dec None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:49.512097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 616/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 7 16:07:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:49.518344 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ed1d1-b8c2-480f-98b2-fe89242f6855 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:49.623718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ed1d1-b8c2-480f-98b2-fe89242f6855 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:49.658503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdbbc9-a8e2-4d84-a27e-c29249a90777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:49.659114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdbbc9-a8e2-4d84-a27e-c29249a90777 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:49.659391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdbbc9-a8e2-4d84-a27e-c29249a90777 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:49.659663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdbbc9-a8e2-4d84-a27e-c29249a90777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:49.713252 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:49.725541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdbbc9-a8e2-4d84-a27e-c29249a90777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:49.726974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccdbbc9-a8e2-4d84-a27e-c29249a90777 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:49.821644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 639/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:07:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:07:49.827495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a56c7-4a89-4db0-b71f-4f938a127dec None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:49.925138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a56c7-4a89-4db0-b71f-4f938a127dec None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:49.939155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:49.961397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183e651-d514-4e5c-8dca-ee3c932a4a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:49.962049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183e651-d514-4e5c-8dca-ee3c932a4a8f None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:49.962320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183e651-d514-4e5c-8dca-ee3c932a4a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:49.962586 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f183e651-d514-4e5c-8dca-ee3c932a4a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:49.972605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183e651-d514-4e5c-8dca-ee3c932a4a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:49.973983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f183e651-d514-4e5c-8dca-ee3c932a4a8f None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-f183e651-d514-4e5c-8dca-ee3c932a4a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:49.988120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:49.989255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:49.990159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:49.990159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:49.990159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:49.990159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 617/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:49.996565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdbbc9-a8e2-4d84-a27e-c29249a90777 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:50.083159 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:50.094751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdbbc9-a8e2-4d84-a27e-c29249a90777 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:50.127873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b81f7-f1b2-4a1c-86b3-1c4376c4156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:50.128475 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b81f7-f1b2-4a1c-86b3-1c4376c4156a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:50.128750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b81f7-f1b2-4a1c-86b3-1c4376c4156a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:50.129019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b81f7-f1b2-4a1c-86b3-1c4376c4156a None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:50.129648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b81f7-f1b2-4a1c-86b3-1c4376c4156a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:50.129947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b81f7-f1b2-4a1c-86b3-1c4376c4156a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:50.131305 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b81f7-f1b2-4a1c-86b3-1c4376c4156a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:50.143295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 640/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:50.148869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183e651-d514-4e5c-8dca-ee3c932a4a8f None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:50.246467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f183e651-d514-4e5c-8dca-ee3c932a4a8f None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:50.280025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c8e15-82be-4d3d-a784-44147679dbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:50.280636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c8e15-82be-4d3d-a784-44147679dbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:50.280914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c8e15-82be-4d3d-a784-44147679dbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:50.281185 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c8e15-82be-4d3d-a784-44147679dbe2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:50.287602 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c8e15-82be-4d3d-a784-44147679dbe2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:50.289060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c8e15-82be-4d3d-a784-44147679dbe2 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-f45c8e15-82be-4d3d-a784-44147679dbe2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:07:50.300851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:50.301917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:50.303059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:07:50.303059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:07:50.303059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:07:50.303059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:07:50.303059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:07:50.303059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:07:50.303059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:50.303059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:50.303059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 618/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:07:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:50.307934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b81f7-f1b2-4a1c-86b3-1c4376c4156a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:50.407522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b81f7-f1b2-4a1c-86b3-1c4376c4156a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:50.441078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5362afe-0c9d-46a7-b8ba-11014d58e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:50.441678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5362afe-0c9d-46a7-b8ba-11014d58e491 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:50.441954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5362afe-0c9d-46a7-b8ba-11014d58e491 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:50.442227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5362afe-0c9d-46a7-b8ba-11014d58e491 None admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:50.442904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5362afe-0c9d-46a7-b8ba-11014d58e491 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:50.443202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5362afe-0c9d-46a7-b8ba-11014d58e491 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:50.444691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5362afe-0c9d-46a7-b8ba-11014d58e491 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:50.456895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 641/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:50.463234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c8e15-82be-4d3d-a784-44147679dbe2 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:50.488798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:50.562478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c8e15-82be-4d3d-a784-44147679dbe2 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:50.596327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01843e5b-e284-4b1c-af79-e3b4d96dba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:50.596919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843e5b-e284-4b1c-af79-e3b4d96dba68 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:50.597201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843e5b-e284-4b1c-af79-e3b4d96dba68 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:50.597476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01843e5b-e284-4b1c-af79-e3b4d96dba68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:50.598195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01843e5b-e284-4b1c-af79-e3b4d96dba68 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=5fdfb0bf09db406ab5c09fbef887cbab) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:50.598503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01843e5b-e284-4b1c-af79-e3b4d96dba68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:50.599959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01843e5b-e284-4b1c-af79-e3b4d96dba68 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:50.605262 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 619/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:07:50 2022] GET /identity/v3/role_assignments?user.id=5fdfb0bf09db406ab5c09fbef887cbab&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:52.721048 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf281c-b147-4dee-8055-7edee5634292 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:52.721579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf281c-b147-4dee-8055-7edee5634292 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:52.721579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf281c-b147-4dee-8055-7edee5634292 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:52.722903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 642/1261] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:07:52.728003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbb2b9-3d03-4d97-b397-49ae5db8974f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:52.728375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbb2b9-3d03-4d97-b397-49ae5db8974f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:52.728548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fbb2b9-3d03-4d97-b397-49ae5db8974f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:52.746924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-e5fbb2b9-3d03-4d97-b397-49ae5db8974f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:52.748140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e5fbb2b9-3d03-4d97-b397-49ae5db8974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:52.815309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 620/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:07:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:07:53.080746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf281c-b147-4dee-8055-7edee5634292 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:53.091474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:53.182035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf281c-b147-4dee-8055-7edee5634292 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:53.214789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66fd14-b9fa-451a-8be1-96a5ad9a6728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:53.215408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66fd14-b9fa-451a-8be1-96a5ad9a6728 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:53.215682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66fd14-b9fa-451a-8be1-96a5ad9a6728 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:53.215975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66fd14-b9fa-451a-8be1-96a5ad9a6728 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:53.226333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66fd14-b9fa-451a-8be1-96a5ad9a6728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-cc66fd14-b9fa-451a-8be1-96a5ad9a6728 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:07:53.231141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:53.240311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:53.241300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 643/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:07:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:53.241997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbb2b9-3d03-4d97-b397-49ae5db8974f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:53.256824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:53.338552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fbb2b9-3d03-4d97-b397-49ae5db8974f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:53.371570 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0af358-f717-4091-97ce-eb5957bc3522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:53.372211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af358-f717-4091-97ce-eb5957bc3522 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:53.372495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af358-f717-4091-97ce-eb5957bc3522 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:53.372772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af358-f717-4091-97ce-eb5957bc3522 None admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:53.373462 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0af358-f717-4091-97ce-eb5957bc3522 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:53.373766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0af358-f717-4091-97ce-eb5957bc3522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:53.375144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0af358-f717-4091-97ce-eb5957bc3522 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:53.383178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 621/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:07:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:53.389035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66fd14-b9fa-451a-8be1-96a5ad9a6728 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:53.455885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:53.487594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66fd14-b9fa-451a-8be1-96a5ad9a6728 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:53.520688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e32410-1c04-4916-8837-ba5d3eb80462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:53.521313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e32410-1c04-4916-8837-ba5d3eb80462 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:53.521593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e32410-1c04-4916-8837-ba5d3eb80462 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:53.521867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e32410-1c04-4916-8837-ba5d3eb80462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:53.531411 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e32410-1c04-4916-8837-ba5d3eb80462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:53.533471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e32410-1c04-4916-8837-ba5d3eb80462 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-10e32410-1c04-4916-8837-ba5d3eb80462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:53.548370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:53.549688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:53.550591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:53.550591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:53.550591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:53.550591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 644/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:53.555549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0af358-f717-4091-97ce-eb5957bc3522 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:53.643124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:53.661701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0af358-f717-4091-97ce-eb5957bc3522 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:53.696317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54bc7f-87d4-4e53-bc6b-104003a34cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:53.696917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54bc7f-87d4-4e53-bc6b-104003a34cbb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:53.697291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54bc7f-87d4-4e53-bc6b-104003a34cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:53.697563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54bc7f-87d4-4e53-bc6b-104003a34cbb None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:53.698190 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54bc7f-87d4-4e53-bc6b-104003a34cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:53.698488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54bc7f-87d4-4e53-bc6b-104003a34cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:53.699855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54bc7f-87d4-4e53-bc6b-104003a34cbb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:53.711359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 622/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:53.717092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e32410-1c04-4916-8837-ba5d3eb80462 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:53.815825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e32410-1c04-4916-8837-ba5d3eb80462 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:53.836819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:53.850120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9ed40-1e64-494a-9997-829c6af484c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:53.850741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9ed40-1e64-494a-9997-829c6af484c8 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:53.851016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9ed40-1e64-494a-9997-829c6af484c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:53.851289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9ed40-1e64-494a-9997-829c6af484c8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:53.858415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9ed40-1e64-494a-9997-829c6af484c8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:53.861224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9ed40-1e64-494a-9997-829c6af484c8 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-6df9ed40-1e64-494a-9997-829c6af484c8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:07:53.871181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:53.872328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:53.873548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:53.873548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:53.873548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:07:53.873548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:07:53.873548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:07:53.873548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:07:53.873548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:07:53.873548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:07:53.873548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:53.873548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:53.873548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 645/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:53.878648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54bc7f-87d4-4e53-bc6b-104003a34cbb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:53.976864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54bc7f-87d4-4e53-bc6b-104003a34cbb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:54.015865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acf6ae-f8de-4499-a53d-3c2523daadec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:54.015865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acf6ae-f8de-4499-a53d-3c2523daadec None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:54.015865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acf6ae-f8de-4499-a53d-3c2523daadec None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:54.015865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acf6ae-f8de-4499-a53d-3c2523daadec None admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:54.016300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acf6ae-f8de-4499-a53d-3c2523daadec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:54.016300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acf6ae-f8de-4499-a53d-3c2523daadec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:54.016300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acf6ae-f8de-4499-a53d-3c2523daadec None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:54.024987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:54.033214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 623/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 7 16:07:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:54.039151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9ed40-1e64-494a-9997-829c6af484c8 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:54.147442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9ed40-1e64-494a-9997-829c6af484c8 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:54.185790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1c419-50d5-4b19-949f-0e7d44010df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:54.186435 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1c419-50d5-4b19-949f-0e7d44010df8 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:54.186761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1c419-50d5-4b19-949f-0e7d44010df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:54.187055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1c419-50d5-4b19-949f-0e7d44010df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:54.199420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1c419-50d5-4b19-949f-0e7d44010df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:54.200922 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1c419-50d5-4b19-949f-0e7d44010df8 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-06f1c419-50d5-4b19-949f-0e7d44010df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:54.215494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:54.216500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:54.217497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:54.217497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:54.217497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:54.217497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 646/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:54.222646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acf6ae-f8de-4499-a53d-3c2523daadec None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:54.324920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acf6ae-f8de-4499-a53d-3c2523daadec None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:54.359593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d901116-d10a-44d6-98da-8d493775943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:54.360859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d901116-d10a-44d6-98da-8d493775943c None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:54.361138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d901116-d10a-44d6-98da-8d493775943c None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:54.361412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d901116-d10a-44d6-98da-8d493775943c None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:54.362063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d901116-d10a-44d6-98da-8d493775943c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:54.362346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d901116-d10a-44d6-98da-8d493775943c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:54.363706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d901116-d10a-44d6-98da-8d493775943c None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:54.374569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 624/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:54.380384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1c419-50d5-4b19-949f-0e7d44010df8 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:54.412474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:54.479895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1c419-50d5-4b19-949f-0e7d44010df8 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:54.513008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026c8ff-e537-42dd-9d96-d451c958b7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:54.513627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026c8ff-e537-42dd-9d96-d451c958b7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:54.513905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026c8ff-e537-42dd-9d96-d451c958b7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:54.514177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026c8ff-e537-42dd-9d96-d451c958b7b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:54.520574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026c8ff-e537-42dd-9d96-d451c958b7b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:54.521992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026c8ff-e537-42dd-9d96-d451c958b7b4 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-d026c8ff-e537-42dd-9d96-d451c958b7b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:07:54.533489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:54.534446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:54.535930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:07:54.535930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:07:54.535930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:07:54.535930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:07:54.535930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:07:54.535930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:07:54.535930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:54.535930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:54.535930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 647/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:07:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:54.540488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d901116-d10a-44d6-98da-8d493775943c None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:54.598744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:54.638280 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d901116-d10a-44d6-98da-8d493775943c None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:54.670682 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2699c52-7e9c-4556-803e-7e9b2877f544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:54.671282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2699c52-7e9c-4556-803e-7e9b2877f544 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:54.671555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2699c52-7e9c-4556-803e-7e9b2877f544 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:54.671850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2699c52-7e9c-4556-803e-7e9b2877f544 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:54.673064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2699c52-7e9c-4556-803e-7e9b2877f544 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:54.673366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2699c52-7e9c-4556-803e-7e9b2877f544 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:54.675059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2699c52-7e9c-4556-803e-7e9b2877f544 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:54.686641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 625/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:07:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:54.692817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026c8ff-e537-42dd-9d96-d451c958b7b4 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:54.781203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:54.790003 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026c8ff-e537-42dd-9d96-d451c958b7b4 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:54.821921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974a220-d950-46d9-8af3-3e995b7c632d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:54.822574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974a220-d950-46d9-8af3-3e995b7c632d None admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:54.822848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974a220-d950-46d9-8af3-3e995b7c632d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:54.823116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974a220-d950-46d9-8af3-3e995b7c632d None admin] PATH_INFO: `/v3/projects/c479846c3d274061a55c19a95ddb5044/users/5fdfb0bf09db406ab5c09fbef887cbab/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:54.824980 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974a220-d950-46d9-8af3-3e995b7c632d None admin] RBAC: Authorizing `identity:create_grant(project_id=c479846c3d274061a55c19a95ddb5044, user_id=5fdfb0bf09db406ab5c09fbef887cbab, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:54.826392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974a220-d950-46d9-8af3-3e995b7c632d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:54.920659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 648/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Wed Sep 7 16:07:54 2022] PUT /identity/v3/projects/c479846c3d274061a55c19a95ddb5044/users/5fdfb0bf09db406ab5c09fbef887cbab/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:57.102437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d980de-c0fd-4336-a952-0a017264d78b None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:57.102954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d980de-c0fd-4336-a952-0a017264d78b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:57.103043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d980de-c0fd-4336-a952-0a017264d78b None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:57.104369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 626/1274] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:07:57.110203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052e1fe-16ee-4661-90af-648f8446ecde None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:57.110634 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052e1fe-16ee-4661-90af-648f8446ecde None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:57.110757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052e1fe-16ee-4661-90af-648f8446ecde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:57.130152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8052e1fe-16ee-4661-90af-648f8446ecde None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:57.131972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8052e1fe-16ee-4661-90af-648f8446ecde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:57.210307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 649/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:07:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:07:57.480166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d980de-c0fd-4336-a952-0a017264d78b None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:57.608756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d980de-c0fd-4336-a952-0a017264d78b None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:57.629895 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:57.647524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c285cf7-6321-40f8-81a6-38b711ad6001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:57.648238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285cf7-6321-40f8-81a6-38b711ad6001 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:57.648515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285cf7-6321-40f8-81a6-38b711ad6001 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:57.648786 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c285cf7-6321-40f8-81a6-38b711ad6001 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:57.660925 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c285cf7-6321-40f8-81a6-38b711ad6001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-0c285cf7-6321-40f8-81a6-38b711ad6001 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 16:07:57.665806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:57.666973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:57.667991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 627/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Wed Sep 7 16:07:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:57.673677 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052e1fe-16ee-4661-90af-648f8446ecde None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:57.761058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:57.781293 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052e1fe-16ee-4661-90af-648f8446ecde None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:57.818986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414ab37-2f04-4708-9bb1-8a633adb0b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:57.819606 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414ab37-2f04-4708-9bb1-8a633adb0b4a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:57.819924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414ab37-2f04-4708-9bb1-8a633adb0b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:57.820203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414ab37-2f04-4708-9bb1-8a633adb0b4a None admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:57.820886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414ab37-2f04-4708-9bb1-8a633adb0b4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:57.821192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414ab37-2f04-4708-9bb1-8a633adb0b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:57.822658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414ab37-2f04-4708-9bb1-8a633adb0b4a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:57.832568 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 650/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Wed Sep 7 16:07:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:07:57.838567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c285cf7-6321-40f8-81a6-38b711ad6001 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:57.942888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c285cf7-6321-40f8-81a6-38b711ad6001 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:57.977566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce2ee9-002d-4436-86c0-70bc3d692003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:57.978205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce2ee9-002d-4436-86c0-70bc3d692003 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:57.978485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce2ee9-002d-4436-86c0-70bc3d692003 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:57.978755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce2ee9-002d-4436-86c0-70bc3d692003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:57.989502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce2ee9-002d-4436-86c0-70bc3d692003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:57.990958 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce2ee9-002d-4436-86c0-70bc3d692003 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-8cce2ee9-002d-4436-86c0-70bc3d692003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:58.005604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:58.006801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:58.008007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:58.008007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:58.008007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:58.008007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 628/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:58.012632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414ab37-2f04-4708-9bb1-8a633adb0b4a None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:58.116767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414ab37-2f04-4708-9bb1-8a633adb0b4a None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:58.150239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb2f9d-abcf-495a-a1fc-e11922eb3c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:58.150864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb2f9d-abcf-495a-a1fc-e11922eb3c28 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:58.151138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb2f9d-abcf-495a-a1fc-e11922eb3c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:58.151402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb2f9d-abcf-495a-a1fc-e11922eb3c28 None admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:58.152059 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb2f9d-abcf-495a-a1fc-e11922eb3c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:58.152405 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb2f9d-abcf-495a-a1fc-e11922eb3c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:58.153757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb2f9d-abcf-495a-a1fc-e11922eb3c28 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:58.165386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 651/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:58.171274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce2ee9-002d-4436-86c0-70bc3d692003 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:58.202207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:58.268763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce2ee9-002d-4436-86c0-70bc3d692003 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:58.302074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5586e4-0924-4143-9682-052a322eecdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:58.302690 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5586e4-0924-4143-9682-052a322eecdb None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:58.302966 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5586e4-0924-4143-9682-052a322eecdb None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:58.303236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5586e4-0924-4143-9682-052a322eecdb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:58.310269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5586e4-0924-4143-9682-052a322eecdb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:58.311688 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5586e4-0924-4143-9682-052a322eecdb None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-4f5586e4-0924-4143-9682-052a322eecdb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 16:07:58.323528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:58.324686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:58.325894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:58.325894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:58.325894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 16:07:58.325894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 16:07:58.325894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 16:07:58.325894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 16:07:58.325894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 16:07:58.325894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 16:07:58.325894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 16:07:58.325894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:07:58.325894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 629/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:58.331074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb2f9d-abcf-495a-a1fc-e11922eb3c28 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:58.334798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:58.428778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb2f9d-abcf-495a-a1fc-e11922eb3c28 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:58.462436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11ec59-84ac-4c8a-add6-2064dece82e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:58.463035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11ec59-84ac-4c8a-add6-2064dece82e5 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:58.463312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11ec59-84ac-4c8a-add6-2064dece82e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:58.463582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11ec59-84ac-4c8a-add6-2064dece82e5 None admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:58.464331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11ec59-84ac-4c8a-add6-2064dece82e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:58.464634 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11ec59-84ac-4c8a-add6-2064dece82e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:58.466105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11ec59-84ac-4c8a-add6-2064dece82e5 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:58.482619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 652/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Wed Sep 7 16:07:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:58.488995 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5586e4-0924-4143-9682-052a322eecdb None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:58.579238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:58.588386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5586e4-0924-4143-9682-052a322eecdb None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:58.621429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bf331-7248-40b4-801a-9a98721d0bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:58.622067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf331-7248-40b4-801a-9a98721d0bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:58.622431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf331-7248-40b4-801a-9a98721d0bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:58.622994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659bf331-7248-40b4-801a-9a98721d0bd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:58.690179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bf331-7248-40b4-801a-9a98721d0bd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:58.691625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659bf331-7248-40b4-801a-9a98721d0bd5 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:58.778019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:58.787859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 630/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Wed Sep 7 16:07:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 16:07:58.793658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11ec59-84ac-4c8a-add6-2064dece82e5 None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:58.853868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:58.889932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11ec59-84ac-4c8a-add6-2064dece82e5 None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:58.923524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaa4e2-b408-4c0f-9124-b1df25c57e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:58.924430 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaa4e2-b408-4c0f-9124-b1df25c57e7d None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:58.924708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaa4e2-b408-4c0f-9124-b1df25c57e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:58.924984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaa4e2-b408-4c0f-9124-b1df25c57e7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:58.935375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebaa4e2-b408-4c0f-9124-b1df25c57e7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:58.936842 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebaa4e2-b408-4c0f-9124-b1df25c57e7d None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-4ebaa4e2-b408-4c0f-9124-b1df25c57e7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 16:07:58.951270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:58.956060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 16:07:58.957173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 16:07:58.957173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 16:07:58.957173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:58.957173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 653/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:58.958119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bf331-7248-40b4-801a-9a98721d0bd5 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:59.057931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659bf331-7248-40b4-801a-9a98721d0bd5 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:59.091308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5f207-8777-4888-a8e7-9eedb0860d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:59.091951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f207-8777-4888-a8e7-9eedb0860d09 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:59.092238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f207-8777-4888-a8e7-9eedb0860d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:59.092514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f207-8777-4888-a8e7-9eedb0860d09 None admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:59.093149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5f207-8777-4888-a8e7-9eedb0860d09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:59.093450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5f207-8777-4888-a8e7-9eedb0860d09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:59.095303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5f207-8777-4888-a8e7-9eedb0860d09 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:59.108637 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 631/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Wed Sep 7 16:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:07:59.112731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaa4e2-b408-4c0f-9124-b1df25c57e7d None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:59.209136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaa4e2-b408-4c0f-9124-b1df25c57e7d None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:59.222542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:59.242105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe9311-39c5-462a-8f64-0ce0f0353afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:59.242721 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe9311-39c5-462a-8f64-0ce0f0353afd None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:59.242999 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe9311-39c5-462a-8f64-0ce0f0353afd None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:59.243271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe9311-39c5-462a-8f64-0ce0f0353afd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:59.249979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe9311-39c5-462a-8f64-0ce0f0353afd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:59.251466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe9311-39c5-462a-8f64-0ce0f0353afd None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application [None req-d2fe9311-39c5-462a-8f64-0ce0f0353afd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 16:07:59.262491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application with Lock( Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:07:59.263561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:07:59.264543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 16:07:59.264543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 16:07:59.264543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 16:07:59.264543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 16:07:59.264543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 16:07:59.264543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 16:07:59.264543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 16:07:59.264543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: ERROR keystone.server.flask.application Sep 07 16:07:59.264543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 654/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Wed Sep 7 16:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:07:59.269990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5f207-8777-4888-a8e7-9eedb0860d09 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:59.319664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:59.368613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5f207-8777-4888-a8e7-9eedb0860d09 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:59.401696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f429af-7a67-43be-8b6e-7be3bfa5e837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:59.402300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f429af-7a67-43be-8b6e-7be3bfa5e837 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:59.402578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f429af-7a67-43be-8b6e-7be3bfa5e837 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:59.402850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f429af-7a67-43be-8b6e-7be3bfa5e837 None admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:59.403531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f429af-7a67-43be-8b6e-7be3bfa5e837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:59.403862 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f429af-7a67-43be-8b6e-7be3bfa5e837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:59.405312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f429af-7a67-43be-8b6e-7be3bfa5e837 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:59.417787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 632/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Wed Sep 7 16:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:59.423887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe9311-39c5-462a-8f64-0ce0f0353afd None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:59.522965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe9311-39c5-462a-8f64-0ce0f0353afd None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:59.555980 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6998e5a-850c-4f16-a8c0-45f2778268e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:59.556564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6998e5a-850c-4f16-a8c0-45f2778268e7 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:59.556839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6998e5a-850c-4f16-a8c0-45f2778268e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:59.557109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6998e5a-850c-4f16-a8c0-45f2778268e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:59.557824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6998e5a-850c-4f16-a8c0-45f2778268e7 None admin] RBAC: Adding query filter params (role.id=4f1f6a27bf0e4b7190e52bcebb14bf70, scope.project.id=c479846c3d274061a55c19a95ddb5044, user.id=5fdfb0bf09db406ab5c09fbef887cbab) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:59.558128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6998e5a-850c-4f16-a8c0-45f2778268e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:59.559538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6998e5a-850c-4f16-a8c0-45f2778268e7 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:59.565248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 655/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Wed Sep 7 16:07:59 2022] GET /identity/v3/role_assignments?user.id=5fdfb0bf09db406ab5c09fbef887cbab&scope.project.id=c479846c3d274061a55c19a95ddb5044&role.id=4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:08:01.759309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9df73-3b09-40dc-8e78-d78a6419fc8e None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:01.759931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9df73-3b09-40dc-8e78-d78a6419fc8e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:01.759931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9df73-3b09-40dc-8e78-d78a6419fc8e None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:01.761222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 633/1288] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:08:01.766395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a73ed-0654-41dd-b1a3-093b310a4772 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:01.766812 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a73ed-0654-41dd-b1a3-093b310a4772 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:01.766940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a73ed-0654-41dd-b1a3-093b310a4772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:01.784868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-4d2a73ed-0654-41dd-b1a3-093b310a4772 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:08:01.786064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4d2a73ed-0654-41dd-b1a3-093b310a4772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:01.815614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:01.850992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 656/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:08:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:08:02.130038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9df73-3b09-40dc-8e78-d78a6419fc8e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:02.229724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9df73-3b09-40dc-8e78-d78a6419fc8e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:02.243178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:02.270736 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1df45c-c116-453e-8e7f-9a67e42754d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:02.273452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1df45c-c116-453e-8e7f-9a67e42754d1 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:02.273452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1df45c-c116-453e-8e7f-9a67e42754d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:02.273452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1df45c-c116-453e-8e7f-9a67e42754d1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:02.273452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1df45c-c116-453e-8e7f-9a67e42754d1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:02.274416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1df45c-c116-453e-8e7f-9a67e42754d1 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-ae1df45c-c116-453e-8e7f-9a67e42754d1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:08:02.288980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:08:02.290211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 16:08:02.291216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:08:02.295921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 634/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 7 16:08:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:08:02.299629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a73ed-0654-41dd-b1a3-093b310a4772 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:02.401044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a73ed-0654-41dd-b1a3-093b310a4772 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:02.434579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f7b80-7ca2-4c7f-99db-62c22d726e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:02.435175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f7b80-7ca2-4c7f-99db-62c22d726e3a None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:02.435452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f7b80-7ca2-4c7f-99db-62c22d726e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:02.435724 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f7b80-7ca2-4c7f-99db-62c22d726e3a None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:02.436381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f7b80-7ca2-4c7f-99db-62c22d726e3a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:08:02.436679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f7b80-7ca2-4c7f-99db-62c22d726e3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:02.438034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f7b80-7ca2-4c7f-99db-62c22d726e3a None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:02.442616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 657/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Sep 7 16:08:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:08:02.448415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1df45c-c116-453e-8e7f-9a67e42754d1 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:02.545461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1df45c-c116-453e-8e7f-9a67e42754d1 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:02.577689 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d89fa-aa39-4718-8be7-970df1d3c765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:02.578287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d89fa-aa39-4718-8be7-970df1d3c765 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:02.579131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d89fa-aa39-4718-8be7-970df1d3c765 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:02.579404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d89fa-aa39-4718-8be7-970df1d3c765 None admin] PATH_INFO: `/v3/services` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:02.580205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d89fa-aa39-4718-8be7-970df1d3c765 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:08:02.580670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d89fa-aa39-4718-8be7-970df1d3c765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:02.582890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d89fa-aa39-4718-8be7-970df1d3c765 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:02.588268 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 635/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Sep 7 16:08:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:08:04.730962 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4cbdd-f3ad-45c5-ad74-e9b242e64af2 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:04.731516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4cbdd-f3ad-45c5-ad74-e9b242e64af2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:04.731516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4cbdd-f3ad-45c5-ad74-e9b242e64af2 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:04.732884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 658/1293] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:08:04.738211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd22f-1b96-4bcd-96aa-deb864baf883 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:04.738615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd22f-1b96-4bcd-96aa-deb864baf883 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:04.738761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fd22f-1b96-4bcd-96aa-deb864baf883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:04.758067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-088fd22f-1b96-4bcd-96aa-deb864baf883 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:08:04.759256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-088fd22f-1b96-4bcd-96aa-deb864baf883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:04.789125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:04.825271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 636/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:08:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:08:05.096688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4cbdd-f3ad-45c5-ad74-e9b242e64af2 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:05.127291 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:05.194491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4cbdd-f3ad-45c5-ad74-e9b242e64af2 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:05.228069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64519796-d684-4d3f-a587-cf7fe88964a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:05.228663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519796-d684-4d3f-a587-cf7fe88964a0 None admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:05.228940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519796-d684-4d3f-a587-cf7fe88964a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:05.229211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64519796-d684-4d3f-a587-cf7fe88964a0 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:05.229849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64519796-d684-4d3f-a587-cf7fe88964a0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:05.231195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64519796-d684-4d3f-a587-cf7fe88964a0 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:05.315236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 659/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Wed Sep 7 16:08:05 2022] POST /identity/v3/services => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:08:07.465863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917733c-e41b-4005-9807-c4c86581b99f None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:07.466515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917733c-e41b-4005-9807-c4c86581b99f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:07.466515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917733c-e41b-4005-9807-c4c86581b99f None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:07.467783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 637/1296] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:08:07.473575 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35bf08-113b-46e2-b7b9-6579d1a47116 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:07.473992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35bf08-113b-46e2-b7b9-6579d1a47116 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:07.474714 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35bf08-113b-46e2-b7b9-6579d1a47116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:07.492135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6c35bf08-113b-46e2-b7b9-6579d1a47116 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:08:07.493494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6c35bf08-113b-46e2-b7b9-6579d1a47116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:07.565110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 660/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:08:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:08:07.841897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917733c-e41b-4005-9807-c4c86581b99f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:07.943262 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917733c-e41b-4005-9807-c4c86581b99f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:07.977441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8774e-2bde-4f19-8c35-2db842ecb017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:07.978062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8774e-2bde-4f19-8c35-2db842ecb017 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:07.978340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8774e-2bde-4f19-8c35-2db842ecb017 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:07.978613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8774e-2bde-4f19-8c35-2db842ecb017 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:07.979197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8774e-2bde-4f19-8c35-2db842ecb017 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:07.980583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8774e-2bde-4f19-8c35-2db842ecb017 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-71b8774e-2bde-4f19-8c35-2db842ecb017 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:08:07.989149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:08:07.990334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:08:07.991560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 638/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 7 16:08:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:08:07.996226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35bf08-113b-46e2-b7b9-6579d1a47116 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:08.015070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:08.092711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35bf08-113b-46e2-b7b9-6579d1a47116 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:08.124428 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74613e1c-3b43-444f-aaee-a9b91e14345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:08.128349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74613e1c-3b43-444f-aaee-a9b91e14345b None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:08.128349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74613e1c-3b43-444f-aaee-a9b91e14345b None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:08.128349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74613e1c-3b43-444f-aaee-a9b91e14345b None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:08.128349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74613e1c-3b43-444f-aaee-a9b91e14345b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:08:08.128349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74613e1c-3b43-444f-aaee-a9b91e14345b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:08.128349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74613e1c-3b43-444f-aaee-a9b91e14345b None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:08.133211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 661/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Sep 7 16:08:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:08:08.139261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8774e-2bde-4f19-8c35-2db842ecb017 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:08.170878 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:08.237709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8774e-2bde-4f19-8c35-2db842ecb017 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:08.270280 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ebc18a-2a53-41b3-9424-5b03c5f0d573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:08.270872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebc18a-2a53-41b3-9424-5b03c5f0d573 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:08.271150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebc18a-2a53-41b3-9424-5b03c5f0d573 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:08.271420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ebc18a-2a53-41b3-9424-5b03c5f0d573 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:08.272141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebc18a-2a53-41b3-9424-5b03c5f0d573 None admin] RBAC: Adding query filter params (interface=public, service_id=2eaf0acc4049490590904a87e21b91b9, region_id=RegionOne) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:08:08.272449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebc18a-2a53-41b3-9424-5b03c5f0d573 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:08.273867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ebc18a-2a53-41b3-9424-5b03c5f0d573 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:08.278674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 639/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Wed Sep 7 16:08:08 2022] GET /identity/v3/endpoints?service_id=2eaf0acc4049490590904a87e21b91b9&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:08:08.284468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74613e1c-3b43-444f-aaee-a9b91e14345b None admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:08.368106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:08.385840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74613e1c-3b43-444f-aaee-a9b91e14345b None admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:08.419302 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b8402-c4af-4a42-b521-c8246a9ef0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:08.419946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b8402-c4af-4a42-b521-c8246a9ef0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:08.420228 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b8402-c4af-4a42-b521-c8246a9ef0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:08.420501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b8402-c4af-4a42-b521-c8246a9ef0a6 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:08.421099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b8402-c4af-4a42-b521-c8246a9ef0a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:08.422959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b8402-c4af-4a42-b521-c8246a9ef0a6 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:08.429946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 662/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Wed Sep 7 16:08:08 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:08:10.529386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e45f1-0895-4b76-9450-7bdf08c952a1 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:10.529913 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e45f1-0895-4b76-9450-7bdf08c952a1 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:10.530025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e45f1-0895-4b76-9450-7bdf08c952a1 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:10.531910 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 640/1302] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:08:10.537082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e43ff-f98d-4dcf-8a7b-966270ecc018 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:10.537522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e43ff-f98d-4dcf-8a7b-966270ecc018 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:10.537656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e43ff-f98d-4dcf-8a7b-966270ecc018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:10.555655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-f44e43ff-f98d-4dcf-8a7b-966270ecc018 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:08:10.556863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f44e43ff-f98d-4dcf-8a7b-966270ecc018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:10.621182 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 663/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:08:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 16:08:10.884550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e45f1-0895-4b76-9450-7bdf08c952a1 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:10.979967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:10.986692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e45f1-0895-4b76-9450-7bdf08c952a1 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:11.020791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4587fd3-6491-4594-af09-12c5732e34a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:11.021417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4587fd3-6491-4594-af09-12c5732e34a0 None admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:11.021738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4587fd3-6491-4594-af09-12c5732e34a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:11.022012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4587fd3-6491-4594-af09-12c5732e34a0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:11.022592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4587fd3-6491-4594-af09-12c5732e34a0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:11.023976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4587fd3-6491-4594-af09-12c5732e34a0 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application [None req-a4587fd3-6491-4594-af09-12c5732e34a0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 16:08:11.032594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 16:08:11.033785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application with Lock( Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self._enter() Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application return self.creator() Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application created_value = creator( Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: ERROR keystone.server.flask.application Sep 07 16:08:11.034660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 641/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Wed Sep 7 16:08:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 16:08:11.039819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e43ff-f98d-4dcf-8a7b-966270ecc018 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:11.114468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:11.136780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e43ff-f98d-4dcf-8a7b-966270ecc018 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:11.169916 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3a349-b957-4ad7-af53-cc6fcd597bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:11.170514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3a349-b957-4ad7-af53-cc6fcd597bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:11.170788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3a349-b957-4ad7-af53-cc6fcd597bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:11.171087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3a349-b957-4ad7-af53-cc6fcd597bd1 None admin] PATH_INFO: `/v3/services` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:11.171729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3a349-b957-4ad7-af53-cc6fcd597bd1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:08:11.172061 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3a349-b957-4ad7-af53-cc6fcd597bd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:11.173548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3a349-b957-4ad7-af53-cc6fcd597bd1 None admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:11.178801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 664/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Wed Sep 7 16:08:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:08:11.184729 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4587fd3-6491-4594-af09-12c5732e34a0 None admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:11.282453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4587fd3-6491-4594-af09-12c5732e34a0 None admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:11.316287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72ba60-7fe9-407d-a641-f2526cd10314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:11.316882 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72ba60-7fe9-407d-a641-f2526cd10314 None admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:11.317159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72ba60-7fe9-407d-a641-f2526cd10314 None admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:11.317429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72ba60-7fe9-407d-a641-f2526cd10314 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:11.318095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72ba60-7fe9-407d-a641-f2526cd10314 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:11.319441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72ba60-7fe9-407d-a641-f2526cd10314 None admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:11.372827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 642/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:08:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:08:21.175813 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=30 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:21.368574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:00.003450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10259aef-1d94-4bfe-a874-ffc619def030 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:00.004073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10259aef-1d94-4bfe-a874-ffc619def030 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:00.004073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10259aef-1d94-4bfe-a874-ffc619def030 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:00.007316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 665/1307] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:00.012968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7606-1873-4349-9acb-a8f4c4430fc5 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:00.013412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7606-1873-4349-9acb-a8f4c4430fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:00.013622 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7606-1873-4349-9acb-a8f4c4430fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:00.112925 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-211f7606-1873-4349-9acb-a8f4c4430fc5 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:00.114241 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-211f7606-1873-4349-9acb-a8f4c4430fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:00.231075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 643/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:00.537700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3172b-d278-41a9-8c84-9d98616f08cc None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:00.537700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3172b-d278-41a9-8c84-9d98616f08cc None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:00.537700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d3172b-d278-41a9-8c84-9d98616f08cc None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:00.539374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 666/1309] 10.222.0.15 () {54 vars in 965 bytes} [Wed Sep 7 16:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:00.546325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29be52-123c-4a33-b374-f5a245b9efe6 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:00.546846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29be52-123c-4a33-b374-f5a245b9efe6 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:00.547745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29be52-123c-4a33-b374-f5a245b9efe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:00.633890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ef29be52-123c-4a33-b374-f5a245b9efe6 None None] MFA Rules not processed for user `b6a6b52462624e7f84f479144353b51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:00.635233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ef29be52-123c-4a33-b374-f5a245b9efe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:00.658412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:00.719832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 644/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Wed Sep 7 16:14:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:00.728586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3172b-d278-41a9-8c84-9d98616f08cc None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:00.730342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-32d3172b-d278-41a9-8c84-9d98616f08cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:00.847716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d3172b-d278-41a9-8c84-9d98616f08cc None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:00.932336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79772984-f73a-40d1-8ce7-062681856ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:00.932956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79772984-f73a-40d1-8ce7-062681856ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:00.933242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79772984-f73a-40d1-8ce7-062681856ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:00.933516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79772984-f73a-40d1-8ce7-062681856ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:00.935148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-79772984-f73a-40d1-8ce7-062681856ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:01.002820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:01.068016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79772984-f73a-40d1-8ce7-062681856ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:01.069603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79772984-f73a-40d1-8ce7-062681856ce2 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:01.092608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 667/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:03.587328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d8f42-640c-4fed-9200-0427969bb86c None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:03.589858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d8f42-640c-4fed-9200-0427969bb86c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:03.589858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d8f42-640c-4fed-9200-0427969bb86c None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:03.589858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 645/1312] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:03.594621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110123b-fb17-46fe-9890-73a80a4b9f67 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:03.595019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110123b-fb17-46fe-9890-73a80a4b9f67 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:03.595141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b110123b-fb17-46fe-9890-73a80a4b9f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:03.665531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b110123b-fb17-46fe-9890-73a80a4b9f67 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:03.666730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b110123b-fb17-46fe-9890-73a80a4b9f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:03.762247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 668/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:04.043996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d8f42-640c-4fed-9200-0427969bb86c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:04.045525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-674d8f42-640c-4fed-9200-0427969bb86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:04.147599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d8f42-640c-4fed-9200-0427969bb86c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:04.218585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:04.221066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8ebce-6edf-4e49-a371-d25d87a19bd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:04.221659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8ebce-6edf-4e49-a371-d25d87a19bd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:04.221939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8ebce-6edf-4e49-a371-d25d87a19bd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:04.222211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8ebce-6edf-4e49-a371-d25d87a19bd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:04.223825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d5e8ebce-6edf-4e49-a371-d25d87a19bd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:04.329376 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8ebce-6edf-4e49-a371-d25d87a19bd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:04.330760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8ebce-6edf-4e49-a371-d25d87a19bd4 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:04.349032 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 646/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:07.022985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd55d7b-e462-4bcd-b29b-3d878437ef11 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:07.023452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd55d7b-e462-4bcd-b29b-3d878437ef11 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:07.023546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd55d7b-e462-4bcd-b29b-3d878437ef11 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:07.024959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 669/1315] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:07.030237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f98a59-7479-4e88-820a-642b76f0ab11 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:07.030647 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f98a59-7479-4e88-820a-642b76f0ab11 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:07.030831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f98a59-7479-4e88-820a-642b76f0ab11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:07.095468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-26f98a59-7479-4e88-820a-642b76f0ab11 None None] MFA Rules not processed for user `769b0a726d17470cb59ff5847d0b33a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:07.096676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-26f98a59-7479-4e88-820a-642b76f0ab11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:07.146706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:07.190814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 647/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:07.476657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd55d7b-e462-4bcd-b29b-3d878437ef11 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:07.478199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ecd55d7b-e462-4bcd-b29b-3d878437ef11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:07.488121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:07.580283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd55d7b-e462-4bcd-b29b-3d878437ef11 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:07.652605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e4fb3-62a6-4f05-b6a6-279921447b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:07.653142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e4fb3-62a6-4f05-b6a6-279921447b20 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:07.653424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e4fb3-62a6-4f05-b6a6-279921447b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:07.653701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e4fb3-62a6-4f05-b6a6-279921447b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:07.655292 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-029e4fb3-62a6-4f05-b6a6-279921447b20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:07.719031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:07.764894 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e4fb3-62a6-4f05-b6a6-279921447b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:07.766333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e4fb3-62a6-4f05-b6a6-279921447b20 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:07.785467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 670/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:10.858244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e61690-7007-47f4-b56c-f0663731731e None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:10.858860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e61690-7007-47f4-b56c-f0663731731e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:10.858860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e61690-7007-47f4-b56c-f0663731731e None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:10.860756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 648/1318] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:10.866165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f7b1-289f-4560-a718-7fc85a09a06f None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:10.866594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f7b1-289f-4560-a718-7fc85a09a06f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:10.866713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90f7b1-289f-4560-a718-7fc85a09a06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:10.931149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6b90f7b1-289f-4560-a718-7fc85a09a06f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:10.932381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6b90f7b1-289f-4560-a718-7fc85a09a06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:11.027548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 671/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:11.319748 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e61690-7007-47f4-b56c-f0663731731e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:11.321323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b9e61690-7007-47f4-b56c-f0663731731e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:11.430454 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e61690-7007-47f4-b56c-f0663731731e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:11.502116 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f061d-9940-46cc-9a9d-01e54308ac91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:11.502715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f061d-9940-46cc-9a9d-01e54308ac91 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:11.502994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f061d-9940-46cc-9a9d-01e54308ac91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:11.503267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f061d-9940-46cc-9a9d-01e54308ac91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:11.504916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2b1f061d-9940-46cc-9a9d-01e54308ac91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:11.512148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:11.612988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f061d-9940-46cc-9a9d-01e54308ac91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:11.614419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f061d-9940-46cc-9a9d-01e54308ac91 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:11.631843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 649/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:14.704003 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786c2e1-0987-420d-8f91-9a91622dc7a8 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:14.704530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786c2e1-0987-420d-8f91-9a91622dc7a8 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:14.704609 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5786c2e1-0987-420d-8f91-9a91622dc7a8 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:14.705903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 672/1321] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:14.711261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c131bed-45b0-40cc-9766-c369f5a4da4d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:14.711673 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c131bed-45b0-40cc-9766-c369f5a4da4d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:14.711976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c131bed-45b0-40cc-9766-c369f5a4da4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:14.780109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0c131bed-45b0-40cc-9766-c369f5a4da4d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:14.781411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0c131bed-45b0-40cc-9766-c369f5a4da4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:14.846342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:14.887814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 650/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:15.171620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786c2e1-0987-420d-8f91-9a91622dc7a8 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:15.173746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5786c2e1-0987-420d-8f91-9a91622dc7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:15.194054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:15.276718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5786c2e1-0987-420d-8f91-9a91622dc7a8 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:15.350854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676cd76-3613-45fd-98bb-aa7a98650bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:15.351464 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676cd76-3613-45fd-98bb-aa7a98650bca service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:15.351744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676cd76-3613-45fd-98bb-aa7a98650bca service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:15.352049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676cd76-3613-45fd-98bb-aa7a98650bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:15.353671 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f676cd76-3613-45fd-98bb-aa7a98650bca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:15.425006 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:15.462595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676cd76-3613-45fd-98bb-aa7a98650bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:15.464050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676cd76-3613-45fd-98bb-aa7a98650bca service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:15.481511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 673/1323] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:17.664900 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9ae98-0042-4eed-a91e-3a4aced27df7 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:17.665445 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9ae98-0042-4eed-a91e-3a4aced27df7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:17.665445 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9ae98-0042-4eed-a91e-3a4aced27df7 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:17.666739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 651/1324] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:17.673442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddb0da-dd24-47bc-9978-e58d8e8374ac None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:17.674025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddb0da-dd24-47bc-9978-e58d8e8374ac None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:17.674025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddb0da-dd24-47bc-9978-e58d8e8374ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:17.743006 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6fddb0da-dd24-47bc-9978-e58d8e8374ac None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:17.744238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6fddb0da-dd24-47bc-9978-e58d8e8374ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:17.840801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 674/1325] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:18.121017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9ae98-0042-4eed-a91e-3a4aced27df7 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:18.122636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e3e9ae98-0042-4eed-a91e-3a4aced27df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:18.231332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9ae98-0042-4eed-a91e-3a4aced27df7 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:18.298802 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14713b-1ff1-40e6-a319-694b1601c92b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:18.299374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14713b-1ff1-40e6-a319-694b1601c92b service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:18.299655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14713b-1ff1-40e6-a319-694b1601c92b service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:18.299962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14713b-1ff1-40e6-a319-694b1601c92b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:18.301556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0e14713b-1ff1-40e6-a319-694b1601c92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:18.319335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:18.412930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14713b-1ff1-40e6-a319-694b1601c92b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:18.414364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14713b-1ff1-40e6-a319-694b1601c92b service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:18.432313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 652/1326] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:20.660427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e833a7-40b4-4b73-aeff-10deeb0220e0 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:20.660974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e833a7-40b4-4b73-aeff-10deeb0220e0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:20.660974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e833a7-40b4-4b73-aeff-10deeb0220e0 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:20.662307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 675/1327] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:20.667436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83663d87-b2f5-4090-abb4-59e1fbc1c403 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:20.667881 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83663d87-b2f5-4090-abb4-59e1fbc1c403 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:20.668070 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83663d87-b2f5-4090-abb4-59e1fbc1c403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:20.735747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-83663d87-b2f5-4090-abb4-59e1fbc1c403 None None] MFA Rules not processed for user `769b0a726d17470cb59ff5847d0b33a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:20.736916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-83663d87-b2f5-4090-abb4-59e1fbc1c403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:20.803018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:20.836090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 653/1328] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:21.123748 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e833a7-40b4-4b73-aeff-10deeb0220e0 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:21.125859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-90e833a7-40b4-4b73-aeff-10deeb0220e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:21.160295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:21.238005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e833a7-40b4-4b73-aeff-10deeb0220e0 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:21.311024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af17b8-94e3-4a61-8a1d-fe003ed66308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:21.311542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af17b8-94e3-4a61-8a1d-fe003ed66308 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:21.311852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af17b8-94e3-4a61-8a1d-fe003ed66308 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:21.312128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af17b8-94e3-4a61-8a1d-fe003ed66308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:21.314474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-74af17b8-94e3-4a61-8a1d-fe003ed66308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:21.396537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:21.436424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af17b8-94e3-4a61-8a1d-fe003ed66308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:21.438455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af17b8-94e3-4a61-8a1d-fe003ed66308 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:21.457303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 676/1329] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:24.027030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e323f1-a4d2-4b32-852b-a5bfdb1f5f91 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:24.027475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e323f1-a4d2-4b32-852b-a5bfdb1f5f91 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:24.027592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e323f1-a4d2-4b32-852b-a5bfdb1f5f91 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:24.028981 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 654/1330] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:24.034199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e0cb1-e6ca-4428-a8ad-34fcb801ccc3 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:24.034579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e0cb1-e6ca-4428-a8ad-34fcb801ccc3 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:24.034759 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e0cb1-e6ca-4428-a8ad-34fcb801ccc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:24.100158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-fd4e0cb1-e6ca-4428-a8ad-34fcb801ccc3 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:24.101372 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fd4e0cb1-e6ca-4428-a8ad-34fcb801ccc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:24.198706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 677/1331] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:24.478164 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e323f1-a4d2-4b32-852b-a5bfdb1f5f91 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:24.481364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f7e323f1-a4d2-4b32-852b-a5bfdb1f5f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:24.582849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e323f1-a4d2-4b32-852b-a5bfdb1f5f91 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:24.654475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2de562-1093-4c13-9b5c-261c21938c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:24.655071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de562-1093-4c13-9b5c-261c21938c50 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:24.655347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de562-1093-4c13-9b5c-261c21938c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:24.655614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2de562-1093-4c13-9b5c-261c21938c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:24.657739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-fc2de562-1093-4c13-9b5c-261c21938c50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:24.684302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:24.764325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2de562-1093-4c13-9b5c-261c21938c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:24.765710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2de562-1093-4c13-9b5c-261c21938c50 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:24.783936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 655/1332] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:26.981196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd501f-181e-40e1-b031-a67c94a76c43 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:26.981704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd501f-181e-40e1-b031-a67c94a76c43 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:26.981704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd501f-181e-40e1-b031-a67c94a76c43 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:26.982992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 678/1333] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:26.988955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687122f-a76c-497e-bdd1-4b9109f0e4cf None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:26.989372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687122f-a76c-497e-bdd1-4b9109f0e4cf None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:26.989536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687122f-a76c-497e-bdd1-4b9109f0e4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:27.055291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-3687122f-a76c-497e-bdd1-4b9109f0e4cf None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:27.056770 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3687122f-a76c-497e-bdd1-4b9109f0e4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:27.129374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:27.150793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 656/1334] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:27.438742 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd501f-181e-40e1-b031-a67c94a76c43 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:27.440321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-75bd501f-181e-40e1-b031-a67c94a76c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:27.478727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:27.541709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd501f-181e-40e1-b031-a67c94a76c43 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:27.612931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47371be-bf95-4a34-ad94-dec4a3598549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:27.613530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47371be-bf95-4a34-ad94-dec4a3598549 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:27.613816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47371be-bf95-4a34-ad94-dec4a3598549 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:27.614081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47371be-bf95-4a34-ad94-dec4a3598549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:27.615648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e47371be-bf95-4a34-ad94-dec4a3598549 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:27.696282 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:27.721711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47371be-bf95-4a34-ad94-dec4a3598549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:27.723541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47371be-bf95-4a34-ad94-dec4a3598549 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:27.741851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 679/1335] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:30.540261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e8f0f-de22-46e6-ae7d-684e0b257a60 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:30.540823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e8f0f-de22-46e6-ae7d-684e0b257a60 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:30.540823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14e8f0f-de22-46e6-ae7d-684e0b257a60 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:30.542204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 657/1336] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:14:30.548225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacfbe3-f990-4cf5-b34f-bb825c8b8f07 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:30.548685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacfbe3-f990-4cf5-b34f-bb825c8b8f07 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:30.548793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacfbe3-f990-4cf5-b34f-bb825c8b8f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:30.614239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-2dacfbe3-f990-4cf5-b34f-bb825c8b8f07 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:30.615435 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2dacfbe3-f990-4cf5-b34f-bb825c8b8f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:30.710747 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 680/1337] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:30.989437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e8f0f-de22-46e6-ae7d-684e0b257a60 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:30.991016 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f14e8f0f-de22-46e6-ae7d-684e0b257a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:31.093732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14e8f0f-de22-46e6-ae7d-684e0b257a60 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:31.165044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f9f42-2f77-4d80-9939-95d896933d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:31.165643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f9f42-2f77-4d80-9939-95d896933d8a service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:31.165920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f9f42-2f77-4d80-9939-95d896933d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:31.166191 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f9f42-2f77-4d80-9939-95d896933d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:31.167800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-694f9f42-2f77-4d80-9939-95d896933d8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:31.201932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:31.274114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f9f42-2f77-4d80-9939-95d896933d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:31.275567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f9f42-2f77-4d80-9939-95d896933d8a service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:31.294177 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 658/1338] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:35.563303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de5d21-f9b3-48e0-a521-618c35108861 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:35.563861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de5d21-f9b3-48e0-a521-618c35108861 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:35.563861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de5d21-f9b3-48e0-a521-618c35108861 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:35.565209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 681/1339] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:35.570396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0e31f-60c9-4a8c-bddc-f1eda5c5d9bf None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:35.570821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0e31f-60c9-4a8c-bddc-f1eda5c5d9bf None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:35.571083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0e31f-60c9-4a8c-bddc-f1eda5c5d9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:35.635937 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-50f0e31f-60c9-4a8c-bddc-f1eda5c5d9bf None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:35.637129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-50f0e31f-60c9-4a8c-bddc-f1eda5c5d9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:35.723674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:35.742030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 659/1340] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:14:36.032331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9612010-162c-4499-998d-c4494ee44919 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:36.032705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9612010-162c-4499-998d-c4494ee44919 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:36.033049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9612010-162c-4499-998d-c4494ee44919 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:36.034497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 682/1341] 10.222.0.15 () {54 vars in 965 bytes} [Wed Sep 7 16:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:36.040509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df0d96-9383-4caf-ae29-8ef8f79997c9 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:36.040799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df0d96-9383-4caf-ae29-8ef8f79997c9 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:36.041075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df0d96-9383-4caf-ae29-8ef8f79997c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:36.117587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-d8df0d96-9383-4caf-ae29-8ef8f79997c9 None None] MFA Rules not processed for user `b6a6b52462624e7f84f479144353b51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:36.118818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d8df0d96-9383-4caf-ae29-8ef8f79997c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:36.196515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 660/1342] 10.222.0.15 () {58 vars in 1062 bytes} [Wed Sep 7 16:14:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:14:36.205572 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9612010-162c-4499-998d-c4494ee44919 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:36.207095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a9612010-162c-4499-998d-c4494ee44919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:36.256725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:36.313353 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9612010-162c-4499-998d-c4494ee44919 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:36.388086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98fc03f-b62e-4006-8b91-8b8cd94f8409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:36.388699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fc03f-b62e-4006-8b91-8b8cd94f8409 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:36.388975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fc03f-b62e-4006-8b91-8b8cd94f8409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:36.389242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98fc03f-b62e-4006-8b91-8b8cd94f8409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:36.390849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a98fc03f-b62e-4006-8b91-8b8cd94f8409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:36.485343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:36.501748 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fc03f-b62e-4006-8b91-8b8cd94f8409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:36.503188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98fc03f-b62e-4006-8b91-8b8cd94f8409 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:36.521327 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 683/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:40.171076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb4b13-396f-4e0c-8d6e-a1224e81b6ce None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:40.171601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb4b13-396f-4e0c-8d6e-a1224e81b6ce None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:40.171601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb4b13-396f-4e0c-8d6e-a1224e81b6ce None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:40.172986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 661/1344] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:40.178111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93271ed-3e9f-4701-af31-4ffd5b3b91a3 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:40.178517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93271ed-3e9f-4701-af31-4ffd5b3b91a3 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:40.178654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93271ed-3e9f-4701-af31-4ffd5b3b91a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:40.258723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-c93271ed-3e9f-4701-af31-4ffd5b3b91a3 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:40.260020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c93271ed-3e9f-4701-af31-4ffd5b3b91a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:40.372041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 684/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:40.656741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb4b13-396f-4e0c-8d6e-a1224e81b6ce None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:40.658294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-77bb4b13-396f-4e0c-8d6e-a1224e81b6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:40.746440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:40.764409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb4b13-396f-4e0c-8d6e-a1224e81b6ce None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:40.838804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60bf73-eeb2-4a5b-8985-cf0ef82774cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:40.838804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60bf73-eeb2-4a5b-8985-cf0ef82774cb service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:40.839129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60bf73-eeb2-4a5b-8985-cf0ef82774cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:40.839397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60bf73-eeb2-4a5b-8985-cf0ef82774cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:40.841071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-db60bf73-eeb2-4a5b-8985-cf0ef82774cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:40.950861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60bf73-eeb2-4a5b-8985-cf0ef82774cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:40.952315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60bf73-eeb2-4a5b-8985-cf0ef82774cb service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:40.971586 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 662/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:45.536004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc8045-df7a-45af-911f-639a95811a1c None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:45.536549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc8045-df7a-45af-911f-639a95811a1c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:45.536549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc8045-df7a-45af-911f-639a95811a1c None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:45.537919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 685/1347] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:45.543645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae46e68-a8d1-4200-ae09-4e7fb0ab1423 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:45.544148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae46e68-a8d1-4200-ae09-4e7fb0ab1423 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:45.546867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae46e68-a8d1-4200-ae09-4e7fb0ab1423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:45.560681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:45.621105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-eae46e68-a8d1-4200-ae09-4e7fb0ab1423 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:45.622399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-eae46e68-a8d1-4200-ae09-4e7fb0ab1423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:45.729729 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 663/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:46.010775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc8045-df7a-45af-911f-639a95811a1c None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:46.012348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e1fc8045-df7a-45af-911f-639a95811a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:46.070666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:46.115511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc8045-df7a-45af-911f-639a95811a1c None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:46.188194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4576d-3ed0-4e63-bcf1-a7966a5a0a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:46.188800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4576d-3ed0-4e63-bcf1-a7966a5a0a1f service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:46.189078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4576d-3ed0-4e63-bcf1-a7966a5a0a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:46.189348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4576d-3ed0-4e63-bcf1-a7966a5a0a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:46.190927 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-42e4576d-3ed0-4e63-bcf1-a7966a5a0a1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:46.295305 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:46.299481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4576d-3ed0-4e63-bcf1-a7966a5a0a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:46.300948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4576d-3ed0-4e63-bcf1-a7966a5a0a1f service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:46.318500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 686/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:51.191141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df048697-c409-486d-855d-eba795ca9fe9 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:51.191560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df048697-c409-486d-855d-eba795ca9fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:51.191843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df048697-c409-486d-855d-eba795ca9fe9 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:51.193456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 664/1350] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:51.198455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b86bbb-64a6-4090-b425-5ad552031b60 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:51.198885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b86bbb-64a6-4090-b425-5ad552031b60 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:51.199064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b86bbb-64a6-4090-b425-5ad552031b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:51.309839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-82b86bbb-64a6-4090-b425-5ad552031b60 None None] MFA Rules not processed for user `5fdfb0bf09db406ab5c09fbef887cbab`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:51.312144 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-82b86bbb-64a6-4090-b425-5ad552031b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:51.461988 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 687/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:14:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:55.727970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=59 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:55.845373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d42b-85df-46f8-86f6-b672ddbf7dc2 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:55.845892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d42b-85df-46f8-86f6-b672ddbf7dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:55.846342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d42b-85df-46f8-86f6-b672ddbf7dc2 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:55.848172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 665/1352] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:55.853316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb963452-7936-4199-bde1-f42a6af4bf37 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:55.853778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb963452-7936-4199-bde1-f42a6af4bf37 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:55.853873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb963452-7936-4199-bde1-f42a6af4bf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:55.908789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:55.936999 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-cb963452-7936-4199-bde1-f42a6af4bf37 None None] MFA Rules not processed for user `5fdfb0bf09db406ab5c09fbef887cbab`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:55.938548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-cb963452-7936-4199-bde1-f42a6af4bf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:56.124351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 688/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:14:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:57.218026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5df02-2c76-4606-b8eb-acedf2aaa02a None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:57.218645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5df02-2c76-4606-b8eb-acedf2aaa02a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:57.218645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5df02-2c76-4606-b8eb-acedf2aaa02a None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:57.219970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 666/1354] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:14:57.225284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65609b1f-4513-4d24-9dbe-87e7e0cdd0d1 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:57.229161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65609b1f-4513-4d24-9dbe-87e7e0cdd0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:57.229161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65609b1f-4513-4d24-9dbe-87e7e0cdd0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:57.304513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-65609b1f-4513-4d24-9dbe-87e7e0cdd0d1 None None] MFA Rules not processed for user `5fdfb0bf09db406ab5c09fbef887cbab`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:57.306023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-65609b1f-4513-4d24-9dbe-87e7e0cdd0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:57.347278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:57.414655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 689/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:14:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:01.041153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2a568-f28c-48f9-8872-c6cbf76a3ad5 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:01.041671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2a568-f28c-48f9-8872-c6cbf76a3ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:01.041671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2a568-f28c-48f9-8872-c6cbf76a3ad5 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:01.043191 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 667/1356] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:01.048863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e2e35-ee3a-4af0-a5ef-baabcd34b378 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:01.049337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e2e35-ee3a-4af0-a5ef-baabcd34b378 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:01.049610 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e2e35-ee3a-4af0-a5ef-baabcd34b378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:01.123021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-091e2e35-ee3a-4af0-a5ef-baabcd34b378 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:01.124414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-091e2e35-ee3a-4af0-a5ef-baabcd34b378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:01.230546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 690/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:01.532698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ed92a-41b0-46fb-abd3-ce4da39e52c2 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:01.533274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ed92a-41b0-46fb-abd3-ce4da39e52c2 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:01.533973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ed92a-41b0-46fb-abd3-ce4da39e52c2 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:01.536255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 668/1358] 10.222.0.15 () {54 vars in 962 bytes} [Wed Sep 7 16:15:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:01.543617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df32bc-6633-4cb6-8303-1151e513881e None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:01.543937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df32bc-6633-4cb6-8303-1151e513881e None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:01.550836 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df32bc-6633-4cb6-8303-1151e513881e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:01.561982 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:01.627577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b4df32bc-6633-4cb6-8303-1151e513881e None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:01.628962 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b4df32bc-6633-4cb6-8303-1151e513881e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:01.757343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 691/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Sep 7 16:15:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:01.768016 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ed92a-41b0-46fb-abd3-ce4da39e52c2 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:01.769819 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-af5ed92a-41b0-46fb-abd3-ce4da39e52c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:01.931405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ed92a-41b0-46fb-abd3-ce4da39e52c2 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:02.011895 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:02.049519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60541a67-eb6e-4b80-a2b4-4be52e5ce08f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:02.050112 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60541a67-eb6e-4b80-a2b4-4be52e5ce08f service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:02.050395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60541a67-eb6e-4b80-a2b4-4be52e5ce08f service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:02.050653 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60541a67-eb6e-4b80-a2b4-4be52e5ce08f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:02.053948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-60541a67-eb6e-4b80-a2b4-4be52e5ce08f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:02.149969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60541a67-eb6e-4b80-a2b4-4be52e5ce08f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:02.151758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60541a67-eb6e-4b80-a2b4-4be52e5ce08f service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:02.171596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 669/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:15:04.533743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49362fe2-745c-4c95-abf9-f57914cf5d09 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:04.534267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49362fe2-745c-4c95-abf9-f57914cf5d09 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:04.534267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49362fe2-745c-4c95-abf9-f57914cf5d09 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:04.535983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 692/1361] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:04.541228 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907f7b9-ab3c-4b05-9257-422ace5e29e1 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:04.541671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907f7b9-ab3c-4b05-9257-422ace5e29e1 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:04.541841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907f7b9-ab3c-4b05-9257-422ace5e29e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:04.623553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-9907f7b9-ab3c-4b05-9257-422ace5e29e1 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:04.624917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9907f7b9-ab3c-4b05-9257-422ace5e29e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:04.664286 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:04.737501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 670/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:05.026650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49362fe2-745c-4c95-abf9-f57914cf5d09 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:05.028298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-49362fe2-745c-4c95-abf9-f57914cf5d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:05.100295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:05.192547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49362fe2-745c-4c95-abf9-f57914cf5d09 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:05.330226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38f2cc-4f47-48d7-87b6-2e4a27c429ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:05.330846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38f2cc-4f47-48d7-87b6-2e4a27c429ed service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:05.331129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38f2cc-4f47-48d7-87b6-2e4a27c429ed service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:05.331406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38f2cc-4f47-48d7-87b6-2e4a27c429ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:05.333023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4e38f2cc-4f47-48d7-87b6-2e4a27c429ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:05.356737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:05.431555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38f2cc-4f47-48d7-87b6-2e4a27c429ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:05.433210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e38f2cc-4f47-48d7-87b6-2e4a27c429ed service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:05.453076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 693/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:08.055947 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29833c8-45d9-48cb-ab75-469be9ce5ddb None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:08.056463 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29833c8-45d9-48cb-ab75-469be9ce5ddb None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:08.056573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29833c8-45d9-48cb-ab75-469be9ce5ddb None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:08.057857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 671/1364] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:08.063878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45367ff7-9d01-4e15-9255-771c25dfde4f None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:08.064315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45367ff7-9d01-4e15-9255-771c25dfde4f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:08.064622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45367ff7-9d01-4e15-9255-771c25dfde4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:08.145644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-45367ff7-9d01-4e15-9255-771c25dfde4f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:08.147028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-45367ff7-9d01-4e15-9255-771c25dfde4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:08.255457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:08.264029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 694/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:08.552751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3afa0e-bb20-4e43-b3e1-1a3171f35602 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:08.553039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3afa0e-bb20-4e43-b3e1-1a3171f35602 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:08.553318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3afa0e-bb20-4e43-b3e1-1a3171f35602 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:08.554767 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 672/1366] 10.222.0.15 () {54 vars in 962 bytes} [Wed Sep 7 16:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:15:08.561766 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba910d-bd86-4f62-9e19-dc0c943e08e7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:08.562184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba910d-bd86-4f62-9e19-dc0c943e08e7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:08.562638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba910d-bd86-4f62-9e19-dc0c943e08e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:08.643220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-f6ba910d-bd86-4f62-9e19-dc0c943e08e7 None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:08.644533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f6ba910d-bd86-4f62-9e19-dc0c943e08e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:08.776716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 695/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Sep 7 16:15:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:08.785719 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3afa0e-bb20-4e43-b3e1-1a3171f35602 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:08.787592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-be3afa0e-bb20-4e43-b3e1-1a3171f35602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:08.945731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3afa0e-bb20-4e43-b3e1-1a3171f35602 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:08.968987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:09.065988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a18e87-ee37-46a7-99f9-777845d93dc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:09.066606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a18e87-ee37-46a7-99f9-777845d93dc0 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:09.066864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a18e87-ee37-46a7-99f9-777845d93dc0 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:09.067052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a18e87-ee37-46a7-99f9-777845d93dc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:09.068701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a0a18e87-ee37-46a7-99f9-777845d93dc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:09.161046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a18e87-ee37-46a7-99f9-777845d93dc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:09.162665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a18e87-ee37-46a7-99f9-777845d93dc0 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:09.182107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 673/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:15:09.317804 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a80e88-d54a-4b0e-9a52-a79bd2ffe058 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:09.318312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a80e88-d54a-4b0e-9a52-a79bd2ffe058 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:09.318412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a80e88-d54a-4b0e-9a52-a79bd2ffe058 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:09.319885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 696/1369] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:09.326022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0985c07-4442-49ae-95b0-f25395644718 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:09.326389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0985c07-4442-49ae-95b0-f25395644718 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:09.326756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0985c07-4442-49ae-95b0-f25395644718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:09.363979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:09.407470 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c0985c07-4442-49ae-95b0-f25395644718 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:09.408747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c0985c07-4442-49ae-95b0-f25395644718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:09.512226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 674/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:09.912627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24233fa-2f9d-4d9c-8ade-4aacee1c2529 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:09.913037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24233fa-2f9d-4d9c-8ade-4aacee1c2529 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:09.913405 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24233fa-2f9d-4d9c-8ade-4aacee1c2529 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:09.914882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 697/1371] 10.222.0.15 () {54 vars in 964 bytes} [Wed Sep 7 16:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:09.920728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87009c7c-219e-4b9e-ba86-755150f97589 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:09.921114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87009c7c-219e-4b9e-ba86-755150f97589 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:09.921492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87009c7c-219e-4b9e-ba86-755150f97589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:10.016068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-87009c7c-219e-4b9e-ba86-755150f97589 None None] MFA Rules not processed for user `9f5f8dfbb9c44e788f110a4f53425344`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:10.017306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-87009c7c-219e-4b9e-ba86-755150f97589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:10.066801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:10.099807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 675/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Wed Sep 7 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:10.108437 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24233fa-2f9d-4d9c-8ade-4aacee1c2529 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:10.110204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e24233fa-2f9d-4d9c-8ade-4aacee1c2529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:10.167395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:10.223641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24233fa-2f9d-4d9c-8ade-4aacee1c2529 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:10.317464 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a0c3d-67c7-4a86-9837-37c68af1c2d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:10.318186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a0c3d-67c7-4a86-9837-37c68af1c2d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:10.318767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a0c3d-67c7-4a86-9837-37c68af1c2d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:10.318767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a0c3d-67c7-4a86-9837-37c68af1c2d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:10.320355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b00a0c3d-67c7-4a86-9837-37c68af1c2d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:10.463887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:10.484934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a0c3d-67c7-4a86-9837-37c68af1c2d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:10.487194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a0c3d-67c7-4a86-9837-37c68af1c2d4 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:10.524210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 698/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:11.654266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598eaf-d2ae-4998-b73e-ddf6b9a2f3a3 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:11.654840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598eaf-d2ae-4998-b73e-ddf6b9a2f3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:11.654840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04598eaf-d2ae-4998-b73e-ddf6b9a2f3a3 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:11.656231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 676/1374] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:11.662217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b2faf-d8f3-4729-babc-c924aa51bd06 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:11.662631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b2faf-d8f3-4729-babc-c924aa51bd06 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:11.662888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b2faf-d8f3-4729-babc-c924aa51bd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:11.730074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8d5b2faf-d8f3-4729-babc-c924aa51bd06 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:11.731296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8d5b2faf-d8f3-4729-babc-c924aa51bd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:11.830091 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 699/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:12.116928 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04598eaf-d2ae-4998-b73e-ddf6b9a2f3a3 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:12.118486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-04598eaf-d2ae-4998-b73e-ddf6b9a2f3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:12.280080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04598eaf-d2ae-4998-b73e-ddf6b9a2f3a3 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:12.333989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:12.399547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7eb21-6477-41f8-b1d0-49581b3be4cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:12.400175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7eb21-6477-41f8-b1d0-49581b3be4cd service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:12.400448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7eb21-6477-41f8-b1d0-49581b3be4cd service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:12.400716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7eb21-6477-41f8-b1d0-49581b3be4cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:12.402303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-56f7eb21-6477-41f8-b1d0-49581b3be4cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:12.493051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7eb21-6477-41f8-b1d0-49581b3be4cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:12.494993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f7eb21-6477-41f8-b1d0-49581b3be4cd service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:12.513818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 677/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:15:14.844793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f64d9-21b4-4381-b7ee-798b6bf4083b None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:14.845404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f64d9-21b4-4381-b7ee-798b6bf4083b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:14.845616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f64d9-21b4-4381-b7ee-798b6bf4083b None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:14.847208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 700/1377] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:14.852220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a26c7-d8c8-4fbf-bd0f-0a4cc65c92b0 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:14.852605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a26c7-d8c8-4fbf-bd0f-0a4cc65c92b0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:14.852826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a26c7-d8c8-4fbf-bd0f-0a4cc65c92b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:14.924649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-823a26c7-d8c8-4fbf-bd0f-0a4cc65c92b0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:14.925847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-823a26c7-d8c8-4fbf-bd0f-0a4cc65c92b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:14.935616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:15.045676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 678/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:15.344290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f64d9-21b4-4381-b7ee-798b6bf4083b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:15.345905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-bb6f64d9-21b4-4381-b7ee-798b6bf4083b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:15.412758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:15.501046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f64d9-21b4-4381-b7ee-798b6bf4083b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:15.617500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6c4c6-78b2-4529-93c3-8d57ed76c923 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:15.618260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c4c6-78b2-4529-93c3-8d57ed76c923 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:15.618689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c4c6-78b2-4529-93c3-8d57ed76c923 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:15.619060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6c4c6-78b2-4529-93c3-8d57ed76c923 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:15.620750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-36c6c4c6-78b2-4529-93c3-8d57ed76c923 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:15.640848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:15.716899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6c4c6-78b2-4529-93c3-8d57ed76c923 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:15.718679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6c4c6-78b2-4529-93c3-8d57ed76c923 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:15.737409 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 701/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:18.085957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd8969-098e-4ef7-b539-6c59a147baf1 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:18.086387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd8969-098e-4ef7-b539-6c59a147baf1 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:18.086614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd8969-098e-4ef7-b539-6c59a147baf1 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:18.088633 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 679/1380] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:18.094010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3886b-4c52-4cff-9865-7ebf80996369 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:18.094546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3886b-4c52-4cff-9865-7ebf80996369 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:18.094546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe3886b-4c52-4cff-9865-7ebf80996369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:18.163310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-efe3886b-4c52-4cff-9865-7ebf80996369 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:18.164434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-efe3886b-4c52-4cff-9865-7ebf80996369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:18.273534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:18.280308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 702/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:18.567708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd8969-098e-4ef7-b539-6c59a147baf1 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:18.569959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e9dd8969-098e-4ef7-b539-6c59a147baf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:18.734996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:18.745783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd8969-098e-4ef7-b539-6c59a147baf1 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:18.897863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f61dcb-237b-444e-9748-cc46c70686b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:18.898699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f61dcb-237b-444e-9748-cc46c70686b9 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:18.899063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f61dcb-237b-444e-9748-cc46c70686b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:18.899428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f61dcb-237b-444e-9748-cc46c70686b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:18.901855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b6f61dcb-237b-444e-9748-cc46c70686b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:19.011709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f61dcb-237b-444e-9748-cc46c70686b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:19.013365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f61dcb-237b-444e-9748-cc46c70686b9 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:19.023508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:19.032027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 680/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:21.366835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f9fb2-0d1f-44fb-a558-b017ff9a1223 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:21.367367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f9fb2-0d1f-44fb-a558-b017ff9a1223 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:21.367532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f9fb2-0d1f-44fb-a558-b017ff9a1223 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:21.369371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 703/1383] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:21.374720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb2b6c-5ee1-41ab-b184-04ac3e7f8ecf None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:21.375160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb2b6c-5ee1-41ab-b184-04ac3e7f8ecf None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:21.375262 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb2b6c-5ee1-41ab-b184-04ac3e7f8ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:21.445058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c1eb2b6c-5ee1-41ab-b184-04ac3e7f8ecf None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:21.446338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c1eb2b6c-5ee1-41ab-b184-04ac3e7f8ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:21.553494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 681/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:21.843743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f9fb2-0d1f-44fb-a558-b017ff9a1223 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:21.845758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-082f9fb2-0d1f-44fb-a558-b017ff9a1223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:21.999404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f9fb2-0d1f-44fb-a558-b017ff9a1223 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:22.071127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:22.130085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f82fc-2e14-4fdd-8162-d6167d10545e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:22.130979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f82fc-2e14-4fdd-8162-d6167d10545e service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:22.131351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f82fc-2e14-4fdd-8162-d6167d10545e service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:22.131810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f82fc-2e14-4fdd-8162-d6167d10545e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:22.134241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-535f82fc-2e14-4fdd-8162-d6167d10545e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:22.251372 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f82fc-2e14-4fdd-8162-d6167d10545e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:22.253030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f82fc-2e14-4fdd-8162-d6167d10545e service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:22.271651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 704/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:24.472956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718c7d8-0d35-42e7-b3cf-0ffd5025b8e8 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:24.473516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718c7d8-0d35-42e7-b3cf-0ffd5025b8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:24.473516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718c7d8-0d35-42e7-b3cf-0ffd5025b8e8 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:24.474858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 682/1386] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:24.480314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd3754-5545-4f24-a71e-6c3d36e91e4b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:24.481277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd3754-5545-4f24-a71e-6c3d36e91e4b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:24.481470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd3754-5545-4f24-a71e-6c3d36e91e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:24.549331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e7fd3754-5545-4f24-a71e-6c3d36e91e4b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:24.550547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e7fd3754-5545-4f24-a71e-6c3d36e91e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:24.583623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:24.647588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 705/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:24.940283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718c7d8-0d35-42e7-b3cf-0ffd5025b8e8 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:24.941973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1718c7d8-0d35-42e7-b3cf-0ffd5025b8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:25.019904 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:25.089508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718c7d8-0d35-42e7-b3cf-0ffd5025b8e8 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:25.203081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7d166-73e3-48de-bb2f-bd38db5ceb65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:25.203600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7d166-73e3-48de-bb2f-bd38db5ceb65 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:25.203919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7d166-73e3-48de-bb2f-bd38db5ceb65 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:25.204213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7d166-73e3-48de-bb2f-bd38db5ceb65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:25.205779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e6b7d166-73e3-48de-bb2f-bd38db5ceb65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:25.244018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:25.295977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7d166-73e3-48de-bb2f-bd38db5ceb65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:25.297664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b7d166-73e3-48de-bb2f-bd38db5ceb65 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:25.316278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 683/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:15:26.058455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecefda-b518-4165-86eb-ab8a2779fb88 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:26.059011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecefda-b518-4165-86eb-ab8a2779fb88 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:26.059011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ecefda-b518-4165-86eb-ab8a2779fb88 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:26.060382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 706/1389] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:26.065888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdedd6a-5390-4888-b7f1-96799973fcf7 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:26.066373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdedd6a-5390-4888-b7f1-96799973fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:26.066653 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdedd6a-5390-4888-b7f1-96799973fcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:26.140154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-4fdedd6a-5390-4888-b7f1-96799973fcf7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:26.141707 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4fdedd6a-5390-4888-b7f1-96799973fcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:26.262633 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 684/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:26.567284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57256f06-bb56-4ba8-93fd-9688cba63e9b None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:26.575174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57256f06-bb56-4ba8-93fd-9688cba63e9b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:26.575174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57256f06-bb56-4ba8-93fd-9688cba63e9b None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:26.575174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 707/1391] 10.222.0.15 () {54 vars in 961 bytes} [Wed Sep 7 16:15:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:26.575486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd139a-c994-466d-8f6e-44d216027779 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:26.575486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd139a-c994-466d-8f6e-44d216027779 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:26.575749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd139a-c994-466d-8f6e-44d216027779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:26.592533 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:26.681162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-e9bd139a-c994-466d-8f6e-44d216027779 None None] MFA Rules not processed for user `0a5d7cf7534941cb857a4ab5867d978e`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:26.684806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e9bd139a-c994-466d-8f6e-44d216027779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:26.763623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 685/1392] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Sep 7 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:15:26.772738 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57256f06-bb56-4ba8-93fd-9688cba63e9b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:26.774403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-57256f06-bb56-4ba8-93fd-9688cba63e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:26.903376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57256f06-bb56-4ba8-93fd-9688cba63e9b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:26.976020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:26.991877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a690c4-2e7d-46d0-84ed-33051660ca95 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5d7cf7534941cb857a4ab5867d978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:26.992519 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a690c4-2e7d-46d0-84ed-33051660ca95 service glance] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:26.992819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a690c4-2e7d-46d0-84ed-33051660ca95 service glance] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:26.993087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a690c4-2e7d-46d0-84ed-33051660ca95 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:26.994703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-02a690c4-2e7d-46d0-84ed-33051660ca95 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:27.118116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a690c4-2e7d-46d0-84ed-33051660ca95 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:27.119617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a690c4-2e7d-46d0-84ed-33051660ca95 service glance] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:27.139347 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 708/1393] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:28.037222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171039-03c7-4276-8621-1c6ba89ecdda None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:28.037689 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171039-03c7-4276-8621-1c6ba89ecdda None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:28.037792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2171039-03c7-4276-8621-1c6ba89ecdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:28.108628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-f2171039-03c7-4276-8621-1c6ba89ecdda None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:28.109883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f2171039-03c7-4276-8621-1c6ba89ecdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:28.117319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f7d22-93ac-47b9-bca4-1215631da434 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:28.117612 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f7d22-93ac-47b9-bca4-1215631da434 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:28.117897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f7d22-93ac-47b9-bca4-1215631da434 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:28.119259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 709/1394] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 16:15:28.124956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85744654-e4b8-46d2-813a-731a4389659d None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:28.125233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85744654-e4b8-46d2-813a-731a4389659d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:28.125517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85744654-e4b8-46d2-813a-731a4389659d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:28.172542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:28.193269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-85744654-e4b8-46d2-813a-731a4389659d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:28.194506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-85744654-e4b8-46d2-813a-731a4389659d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:28.241312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:28.305961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 710/1395] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:28.324000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 686/1396] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:28.370986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813bc0d-63fd-4383-80ad-97073cc5defd None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:28.371931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813bc0d-63fd-4383-80ad-97073cc5defd None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:28.372212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813bc0d-63fd-4383-80ad-97073cc5defd None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:28.373743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 711/1397] 10.222.0.15 () {54 vars in 943 bytes} [Wed Sep 7 16:15:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:28.380107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4811-aa20-44af-8ce7-1749659d5a44 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:28.380423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4811-aa20-44af-8ce7-1749659d5a44 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:28.380695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4811-aa20-44af-8ce7-1749659d5a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:28.465048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-91fa4811-aa20-44af-8ce7-1749659d5a44 None None] MFA Rules not processed for user `69a8796195ff4dfb85655cdd91a0229d`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:28.467039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-91fa4811-aa20-44af-8ce7-1749659d5a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:28.591383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 687/1398] 10.222.0.15 () {58 vars in 1040 bytes} [Wed Sep 7 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:28.597594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813bc0d-63fd-4383-80ad-97073cc5defd None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:28.598984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7813bc0d-63fd-4383-80ad-97073cc5defd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:28.600505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4811-aa20-44af-8ce7-1749659d5a44 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:28.610175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4811-aa20-44af-8ce7-1749659d5a44 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:28.635336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:28.714832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb8bec-c605-4c76-91b7-3a4ba49af49e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:28.715148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb8bec-c605-4c76-91b7-3a4ba49af49e service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:28.715345 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb8bec-c605-4c76-91b7-3a4ba49af49e service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:28.715687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb8bec-c605-4c76-91b7-3a4ba49af49e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:28.717335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4eeb8bec-c605-4c76-91b7-3a4ba49af49e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:28.751237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813bc0d-63fd-4383-80ad-97073cc5defd None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:28.776625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:28.814339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb8bec-c605-4c76-91b7-3a4ba49af49e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:28.815992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb8bec-c605-4c76-91b7-3a4ba49af49e service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:28.824019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 688/1399] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:15:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 16:15:28.872277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1de0e2-1ca9-4774-acb4-32c001d8f528 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:28.872923 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1de0e2-1ca9-4774-acb4-32c001d8f528 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:28.873228 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1de0e2-1ca9-4774-acb4-32c001d8f528 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:28.873498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1de0e2-1ca9-4774-acb4-32c001d8f528 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:28.875139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-de1de0e2-1ca9-4774-acb4-32c001d8f528 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:28.968086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1de0e2-1ca9-4774-acb4-32c001d8f528 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:28.969604 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1de0e2-1ca9-4774-acb4-32c001d8f528 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:28.987532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 712/1400] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:31.355422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2081e0a-bcb6-425e-ab9b-4b1217f6ad93 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:31.355992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2081e0a-bcb6-425e-ab9b-4b1217f6ad93 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:31.355992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2081e0a-bcb6-425e-ab9b-4b1217f6ad93 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:31.361017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 689/1401] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:31.362703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ece5a-6bc3-446b-9f43-04f809fbd7fb None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:31.363120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ece5a-6bc3-446b-9f43-04f809fbd7fb None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:31.363299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ece5a-6bc3-446b-9f43-04f809fbd7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:31.409088 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:31.448260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ff8ece5a-6bc3-446b-9f43-04f809fbd7fb None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:31.449872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ff8ece5a-6bc3-446b-9f43-04f809fbd7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:31.562136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 713/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:15:31.858588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2081e0a-bcb6-425e-ab9b-4b1217f6ad93 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:31.860133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e2081e0a-bcb6-425e-ab9b-4b1217f6ad93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:31.932656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:32.034403 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2081e0a-bcb6-425e-ab9b-4b1217f6ad93 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:32.161081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f090224-c95b-44d8-b5fe-af2f6dddcdce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:32.161691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f090224-c95b-44d8-b5fe-af2f6dddcdce service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:32.162004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f090224-c95b-44d8-b5fe-af2f6dddcdce service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:32.162275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f090224-c95b-44d8-b5fe-af2f6dddcdce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:32.163888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4f090224-c95b-44d8-b5fe-af2f6dddcdce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:32.184238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:32.267384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f090224-c95b-44d8-b5fe-af2f6dddcdce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:32.269107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f090224-c95b-44d8-b5fe-af2f6dddcdce service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:32.288252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 690/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:34.681583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ce255-f0ef-4871-9bac-5f5c3961be18 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:34.682157 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ce255-f0ef-4871-9bac-5f5c3961be18 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:34.682157 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ce255-f0ef-4871-9bac-5f5c3961be18 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:34.683515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 714/1404] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:34.688809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e0980-8bed-4f5d-b9ec-f75cae9f482c None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:34.689233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e0980-8bed-4f5d-b9ec-f75cae9f482c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:34.689415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e0980-8bed-4f5d-b9ec-f75cae9f482c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:34.767351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c38e0980-8bed-4f5d-b9ec-f75cae9f482c None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:34.768620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c38e0980-8bed-4f5d-b9ec-f75cae9f482c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:34.869287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:34.876004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 691/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:35.175595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ce255-f0ef-4871-9bac-5f5c3961be18 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:35.179984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ca8ce255-f0ef-4871-9bac-5f5c3961be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:35.290606 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:35.338143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ce255-f0ef-4871-9bac-5f5c3961be18 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:35.467002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fafaff-5044-402f-94e1-0091f36c0036 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:35.467717 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fafaff-5044-402f-94e1-0091f36c0036 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:35.468089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fafaff-5044-402f-94e1-0091f36c0036 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:35.468370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fafaff-5044-402f-94e1-0091f36c0036 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:35.469961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a4fafaff-5044-402f-94e1-0091f36c0036 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:35.546372 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:35.569027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fafaff-5044-402f-94e1-0091f36c0036 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:35.570779 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fafaff-5044-402f-94e1-0091f36c0036 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:35.593411 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 715/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:37.895752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b67a5-02c3-4521-88a5-bf34b7aee35d None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:37.896316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b67a5-02c3-4521-88a5-bf34b7aee35d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:37.896316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b67a5-02c3-4521-88a5-bf34b7aee35d None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:37.899804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 692/1407] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:15:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:37.906661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac7372-da40-4095-a02e-95dbaa1a0d18 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:37.906661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac7372-da40-4095-a02e-95dbaa1a0d18 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:37.906661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac7372-da40-4095-a02e-95dbaa1a0d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:37.980491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-faac7372-da40-4095-a02e-95dbaa1a0d18 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:37.981702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-faac7372-da40-4095-a02e-95dbaa1a0d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:38.097285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 716/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:38.394439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b67a5-02c3-4521-88a5-bf34b7aee35d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:38.395995 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-156b67a5-02c3-4521-88a5-bf34b7aee35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:38.558070 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b67a5-02c3-4521-88a5-bf34b7aee35d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:38.633507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:38.687091 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d7dc1-7fcb-495b-a4da-5668b7a973d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:38.687905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d7dc1-7fcb-495b-a4da-5668b7a973d3 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:38.688255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d7dc1-7fcb-495b-a4da-5668b7a973d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:38.688587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d7dc1-7fcb-495b-a4da-5668b7a973d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:38.690276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8e5d7dc1-7fcb-495b-a4da-5668b7a973d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:38.789690 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d7dc1-7fcb-495b-a4da-5668b7a973d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:38.791289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d7dc1-7fcb-495b-a4da-5668b7a973d3 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:38.811467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 693/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:48.095670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=79 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:48.808356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=64 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:52.922921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa40e34-b4d7-449c-a94d-ca38a2ea4b91 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:52.923444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa40e34-b4d7-449c-a94d-ca38a2ea4b91 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:52.923444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa40e34-b4d7-449c-a94d-ca38a2ea4b91 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:52.924849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 717/1410] 10.222.0.15 () {54 vars in 918 bytes} [Wed Sep 7 16:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:52.929788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed04fdf-f603-4e4c-996b-ea0495b0a635 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:52.930491 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed04fdf-f603-4e4c-996b-ea0495b0a635 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:52.930590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed04fdf-f603-4e4c-996b-ea0495b0a635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:53.008093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-aed04fdf-f603-4e4c-996b-ea0495b0a635 None None] MFA Rules not processed for user `5fdfb0bf09db406ab5c09fbef887cbab`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:53.009320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-aed04fdf-f603-4e4c-996b-ea0495b0a635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:53.115537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 694/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Sep 7 16:15:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:53.130078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a5dd5-3234-474e-911b-19ba72c8c72c None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:53.130360 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a5dd5-3234-474e-911b-19ba72c8c72c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:53.130633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a5dd5-3234-474e-911b-19ba72c8c72c None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:53.132019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 718/1412] 10.222.0.15 () {54 vars in 918 bytes} [Wed Sep 7 16:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:53.136614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5820425-808c-4e7d-892f-ba45996c6f71 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:53.136898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5820425-808c-4e7d-892f-ba45996c6f71 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:53.137172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5820425-808c-4e7d-892f-ba45996c6f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:53.210033 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:53.212932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-e5820425-808c-4e7d-892f-ba45996c6f71 None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:53.214172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e5820425-808c-4e7d-892f-ba45996c6f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:53.340855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 695/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Sep 7 16:15:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:55.930843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18074ed7-bbc0-4b7e-b47b-91005669191c None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:55.931378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18074ed7-bbc0-4b7e-b47b-91005669191c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:55.931378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18074ed7-bbc0-4b7e-b47b-91005669191c None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:55.933311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 719/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:15:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:55.938529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86655ecd-7d68-4962-b424-bd3311411e58 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:55.938945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86655ecd-7d68-4962-b424-bd3311411e58 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:55.939059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86655ecd-7d68-4962-b424-bd3311411e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:56.004372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-86655ecd-7d68-4962-b424-bd3311411e58 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:56.005603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-86655ecd-7d68-4962-b424-bd3311411e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:56.061317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:56.102042 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 696/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:15:56.399133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18074ed7-bbc0-4b7e-b47b-91005669191c None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:56.401233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-18074ed7-bbc0-4b7e-b47b-91005669191c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:56.556105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18074ed7-bbc0-4b7e-b47b-91005669191c None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:56.634748 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:56.676991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f3ddf-fa37-4e55-987f-730297ca2851 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:56.677523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f3ddf-fa37-4e55-987f-730297ca2851 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:56.677797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f3ddf-fa37-4e55-987f-730297ca2851 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:56.678065 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f3ddf-fa37-4e55-987f-730297ca2851 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:56.679638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d77f3ddf-fa37-4e55-987f-730297ca2851 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:56.767871 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f3ddf-fa37-4e55-987f-730297ca2851 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:56.769462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f3ddf-fa37-4e55-987f-730297ca2851 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:56.787510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 720/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:57.941129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8b98b-a8e8-4a81-ae2e-ce363c7c2c79 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:57.941775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8b98b-a8e8-4a81-ae2e-ce363c7c2c79 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:57.941775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8b98b-a8e8-4a81-ae2e-ce363c7c2c79 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:57.943133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 697/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:57.949183 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab71aa-7646-46ae-a64c-3ea7f311ed31 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:57.949649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab71aa-7646-46ae-a64c-3ea7f311ed31 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:57.949757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dab71aa-7646-46ae-a64c-3ea7f311ed31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:58.024932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-7dab71aa-7646-46ae-a64c-3ea7f311ed31 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:58.026173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7dab71aa-7646-46ae-a64c-3ea7f311ed31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:58.052618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:58.122892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 721/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:15:58.272756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8b98b-a8e8-4a81-ae2e-ce363c7c2c79 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:58.274234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-64e8b98b-a8e8-4a81-ae2e-ce363c7c2c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:58.444008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8b98b-a8e8-4a81-ae2e-ce363c7c2c79 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:58.483917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:58.557346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73c9d7-3a6c-4216-b163-0220b85ff32f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:58.557877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73c9d7-3a6c-4216-b163-0220b85ff32f service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:58.558149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73c9d7-3a6c-4216-b163-0220b85ff32f service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:58.558412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73c9d7-3a6c-4216-b163-0220b85ff32f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:58.560008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9c73c9d7-3a6c-4216-b163-0220b85ff32f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:58.647014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73c9d7-3a6c-4216-b163-0220b85ff32f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:58.648645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73c9d7-3a6c-4216-b163-0220b85ff32f service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:58.667619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 698/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:59.874103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2bd69-0d20-4de7-83bb-b83c0df37130 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:59.874663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2bd69-0d20-4de7-83bb-b83c0df37130 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:59.874663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2bd69-0d20-4de7-83bb-b83c0df37130 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:59.876030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 722/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:15:59.882171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884554-9a7b-4359-bc22-7e1d7f838c70 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:59.882594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884554-9a7b-4359-bc22-7e1d7f838c70 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:59.882703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51884554-9a7b-4359-bc22-7e1d7f838c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:59.937738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:59.959187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-51884554-9a7b-4359-bc22-7e1d7f838c70 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:15:59.961233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-51884554-9a7b-4359-bc22-7e1d7f838c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:00.058039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 699/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:00.207673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2bd69-0d20-4de7-83bb-b83c0df37130 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:00.209262 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fbf2bd69-0d20-4de7-83bb-b83c0df37130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:00.353533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2bd69-0d20-4de7-83bb-b83c0df37130 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:00.365422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:00.468074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f7dd3-0a27-41be-8f59-2a70f486bf4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:00.468678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7dd3-0a27-41be-8f59-2a70f486bf4f service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:00.468954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7dd3-0a27-41be-8f59-2a70f486bf4f service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:00.469227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7dd3-0a27-41be-8f59-2a70f486bf4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:00.470798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-403f7dd3-0a27-41be-8f59-2a70f486bf4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:00.559884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f7dd3-0a27-41be-8f59-2a70f486bf4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:00.561465 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f7dd3-0a27-41be-8f59-2a70f486bf4f service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:00.579452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 723/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:01.695106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2b3b0-a1a3-48e2-80d9-5ab45bb9b1fc None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:01.695106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2b3b0-a1a3-48e2-80d9-5ab45bb9b1fc None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:01.695106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2b3b0-a1a3-48e2-80d9-5ab45bb9b1fc None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:01.695106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 700/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:01.701134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb59cf0-f486-486e-9f61-59304b8241e4 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:01.701562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb59cf0-f486-486e-9f61-59304b8241e4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:01.701686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb59cf0-f486-486e-9f61-59304b8241e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:01.721672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:01.780793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-0cb59cf0-f486-486e-9f61-59304b8241e4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:01.782053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0cb59cf0-f486-486e-9f61-59304b8241e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:01.882018 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 724/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:02.030140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2b3b0-a1a3-48e2-80d9-5ab45bb9b1fc None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:02.031661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-59f2b3b0-a1a3-48e2-80d9-5ab45bb9b1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:02.138075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:02.173471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2b3b0-a1a3-48e2-80d9-5ab45bb9b1fc None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:02.285628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110dea7f-9285-4e87-971d-854fcae00ab9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:02.286158 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dea7f-9285-4e87-971d-854fcae00ab9 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:02.286432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dea7f-9285-4e87-971d-854fcae00ab9 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:02.286702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110dea7f-9285-4e87-971d-854fcae00ab9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:02.288342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-110dea7f-9285-4e87-971d-854fcae00ab9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:02.364310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:02.377639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dea7f-9285-4e87-971d-854fcae00ab9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:02.379271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110dea7f-9285-4e87-971d-854fcae00ab9 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:02.397266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 701/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:03.512821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d82741-c3eb-4269-ad08-631998eff977 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:03.513341 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d82741-c3eb-4269-ad08-631998eff977 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:03.513341 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d82741-c3eb-4269-ad08-631998eff977 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:03.514724 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 725/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:03.520840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5bb3b-ce10-4a8b-9a8e-e79aa75b7624 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:03.521281 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5bb3b-ce10-4a8b-9a8e-e79aa75b7624 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:03.521377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5bb3b-ce10-4a8b-9a8e-e79aa75b7624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:03.595644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-aee5bb3b-ce10-4a8b-9a8e-e79aa75b7624 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:03.596858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-aee5bb3b-ce10-4a8b-9a8e-e79aa75b7624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:03.691629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 702/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:03.841718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d82741-c3eb-4269-ad08-631998eff977 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:03.843323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-18d82741-c3eb-4269-ad08-631998eff977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:03.927160 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:03.987989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d82741-c3eb-4269-ad08-631998eff977 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:04.097529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b5038-bf30-4a63-aa2f-9c9a9930a3bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:04.098135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5038-bf30-4a63-aa2f-9c9a9930a3bf service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:04.098410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5038-bf30-4a63-aa2f-9c9a9930a3bf service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:04.098687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b5038-bf30-4a63-aa2f-9c9a9930a3bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:04.100276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6a5b5038-bf30-4a63-aa2f-9c9a9930a3bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:04.147003 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:04.187925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b5038-bf30-4a63-aa2f-9c9a9930a3bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:04.189496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b5038-bf30-4a63-aa2f-9c9a9930a3bf service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:04.206937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 726/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:05.309479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7402af-777a-4b9e-9795-fe9b8c875cfc None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:05.309992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7402af-777a-4b9e-9795-fe9b8c875cfc None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:05.309992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7402af-777a-4b9e-9795-fe9b8c875cfc None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:05.311305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 703/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:05.317386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636666a-5cec-496d-a28e-6fe64bfce6cf None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:05.317845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636666a-5cec-496d-a28e-6fe64bfce6cf None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:05.318194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636666a-5cec-496d-a28e-6fe64bfce6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:05.393111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e636666a-5cec-496d-a28e-6fe64bfce6cf None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:05.394303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e636666a-5cec-496d-a28e-6fe64bfce6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:05.490393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 727/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:05.640358 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7402af-777a-4b9e-9795-fe9b8c875cfc None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:05.641875 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8b7402af-777a-4b9e-9795-fe9b8c875cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:05.690443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:05.786781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7402af-777a-4b9e-9795-fe9b8c875cfc None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:05.897983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec37297-27eb-4d12-a063-e3b5c13b5dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:05.898512 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37297-27eb-4d12-a063-e3b5c13b5dd3 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:05.898795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37297-27eb-4d12-a063-e3b5c13b5dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:05.899068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37297-27eb-4d12-a063-e3b5c13b5dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:05.901205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-dec37297-27eb-4d12-a063-e3b5c13b5dd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:05.907964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:05.989470 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec37297-27eb-4d12-a063-e3b5c13b5dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:05.991073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec37297-27eb-4d12-a063-e3b5c13b5dd3 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:06.009284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 704/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:07.135868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117693f2-3941-43af-a383-eeee9295ec7c None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:07.136458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117693f2-3941-43af-a383-eeee9295ec7c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:07.136458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117693f2-3941-43af-a383-eeee9295ec7c None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:07.137807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 728/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:07.143972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fd513-81cd-449f-b2c0-c1ed05a476a9 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:07.144450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fd513-81cd-449f-b2c0-c1ed05a476a9 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:07.144563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fd513-81cd-449f-b2c0-c1ed05a476a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:07.224156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-311fd513-81cd-449f-b2c0-c1ed05a476a9 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:07.225388 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-311fd513-81cd-449f-b2c0-c1ed05a476a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:07.293780 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:07.322187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 705/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:07.472201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117693f2-3941-43af-a383-eeee9295ec7c None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:07.473753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-117693f2-3941-43af-a383-eeee9295ec7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:07.492340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:07.618767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117693f2-3941-43af-a383-eeee9295ec7c None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:07.704827 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:07.730486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e8e53-6e2b-4e92-8f36-611ee92e45f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:07.731114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e8e53-6e2b-4e92-8f36-611ee92e45f3 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:07.731419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e8e53-6e2b-4e92-8f36-611ee92e45f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:07.731687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e8e53-6e2b-4e92-8f36-611ee92e45f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:07.733287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8f7e8e53-6e2b-4e92-8f36-611ee92e45f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:07.821281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e8e53-6e2b-4e92-8f36-611ee92e45f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:07.823387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e8e53-6e2b-4e92-8f36-611ee92e45f3 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:07.841409 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 729/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:08.982795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0266fc-c38d-4c9e-a1b0-d96795ab7723 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:08.983342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0266fc-c38d-4c9e-a1b0-d96795ab7723 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:08.983342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0266fc-c38d-4c9e-a1b0-d96795ab7723 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:08.984758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 706/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:08.991007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73913de0-db9a-4f8c-a982-300c6b512770 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:08.991398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73913de0-db9a-4f8c-a982-300c6b512770 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:08.991586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73913de0-db9a-4f8c-a982-300c6b512770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:09.066740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-73913de0-db9a-4f8c-a982-300c6b512770 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:09.067973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-73913de0-db9a-4f8c-a982-300c6b512770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:09.109354 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:09.163391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 730/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:09.322788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0266fc-c38d-4c9e-a1b0-d96795ab7723 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:09.324278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-be0266fc-c38d-4c9e-a1b0-d96795ab7723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:09.468371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0266fc-c38d-4c9e-a1b0-d96795ab7723 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:09.523479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:09.582386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7396fb-8809-48a7-9ec6-e506ad880b2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:09.582929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7396fb-8809-48a7-9ec6-e506ad880b2f service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:09.583210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7396fb-8809-48a7-9ec6-e506ad880b2f service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:09.583484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7396fb-8809-48a7-9ec6-e506ad880b2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:09.585102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7b7396fb-8809-48a7-9ec6-e506ad880b2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:09.673089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7396fb-8809-48a7-9ec6-e506ad880b2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:09.674947 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7396fb-8809-48a7-9ec6-e506ad880b2f service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:09.692865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 707/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:10.795654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe527920-978a-44d4-8ff2-43366920cd53 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:10.796994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe527920-978a-44d4-8ff2-43366920cd53 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:10.796994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe527920-978a-44d4-8ff2-43366920cd53 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:10.797583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 731/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:10.804067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ac931-c61e-41ad-bd65-17dd57cd05f0 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:10.804505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ac931-c61e-41ad-bd65-17dd57cd05f0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:10.804640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ac931-c61e-41ad-bd65-17dd57cd05f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:10.879996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-9b9ac931-c61e-41ad-bd65-17dd57cd05f0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:10.881203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9b9ac931-c61e-41ad-bd65-17dd57cd05f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:10.883171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:10.976099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 708/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:11.126553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe527920-978a-44d4-8ff2-43366920cd53 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:11.128096 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fe527920-978a-44d4-8ff2-43366920cd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:11.275374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe527920-978a-44d4-8ff2-43366920cd53 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:11.302139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:11.386584 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1b6cd1-afad-44cd-96cb-7ee51775ef72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:11.387115 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b6cd1-afad-44cd-96cb-7ee51775ef72 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:11.387396 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b6cd1-afad-44cd-96cb-7ee51775ef72 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:11.387666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1b6cd1-afad-44cd-96cb-7ee51775ef72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:11.389285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-cc1b6cd1-afad-44cd-96cb-7ee51775ef72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:11.477371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b6cd1-afad-44cd-96cb-7ee51775ef72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:11.479008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1b6cd1-afad-44cd-96cb-7ee51775ef72 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:11.496694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 732/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:13.544844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d144d12-9491-4412-b0a5-eb0755f99771 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:13.545398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d144d12-9491-4412-b0a5-eb0755f99771 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:13.545398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d144d12-9491-4412-b0a5-eb0755f99771 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:13.546755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 709/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:13.552104 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900325f-7d82-4bf6-989c-08d2e16d6a90 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:13.552495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900325f-7d82-4bf6-989c-08d2e16d6a90 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:13.552647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900325f-7d82-4bf6-989c-08d2e16d6a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:13.594049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:13.618237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6900325f-7d82-4bf6-989c-08d2e16d6a90 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:13.619971 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6900325f-7d82-4bf6-989c-08d2e16d6a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:13.716271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 733/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:14.007245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d144d12-9491-4412-b0a5-eb0755f99771 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:14.008908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1d144d12-9491-4412-b0a5-eb0755f99771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:14.111916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d144d12-9491-4412-b0a5-eb0755f99771 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:14.142740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:14.184634 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79238c47-e2c0-4c2c-9eda-2ecdc22f336f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:14.185232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79238c47-e2c0-4c2c-9eda-2ecdc22f336f service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:14.185507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79238c47-e2c0-4c2c-9eda-2ecdc22f336f service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:14.185775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79238c47-e2c0-4c2c-9eda-2ecdc22f336f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:14.187356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-79238c47-e2c0-4c2c-9eda-2ecdc22f336f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:14.296823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79238c47-e2c0-4c2c-9eda-2ecdc22f336f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:14.298179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79238c47-e2c0-4c2c-9eda-2ecdc22f336f service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:14.316471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 710/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:16.636034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc306f4-391f-46ea-b4de-9bae2034e146 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:16.636594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc306f4-391f-46ea-b4de-9bae2034e146 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:16.636594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc306f4-391f-46ea-b4de-9bae2034e146 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:16.637911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 734/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:16.643027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52111642-1f0a-4f8e-a11c-cb86c19fa810 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:16.643411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52111642-1f0a-4f8e-a11c-cb86c19fa810 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:16.643562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52111642-1f0a-4f8e-a11c-cb86c19fa810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:16.709814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-52111642-1f0a-4f8e-a11c-cb86c19fa810 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:16.711013 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-52111642-1f0a-4f8e-a11c-cb86c19fa810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:16.724135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:16.806472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 711/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:17.116104 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc306f4-391f-46ea-b4de-9bae2034e146 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:17.117670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fdc306f4-391f-46ea-b4de-9bae2034e146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:17.223183 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc306f4-391f-46ea-b4de-9bae2034e146 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:17.235457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:17.297301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5cdb38-98fc-4b20-a075-cae048ac8a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:17.297846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cdb38-98fc-4b20-a075-cae048ac8a51 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:17.298121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cdb38-98fc-4b20-a075-cae048ac8a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:17.298387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5cdb38-98fc-4b20-a075-cae048ac8a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:17.299990 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1d5cdb38-98fc-4b20-a075-cae048ac8a51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:17.408378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cdb38-98fc-4b20-a075-cae048ac8a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:17.409854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5cdb38-98fc-4b20-a075-cae048ac8a51 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:17.428021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 735/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:19.955745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b51f64-79ac-4b45-bf67-3556ca88c670 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:19.956341 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b51f64-79ac-4b45-bf67-3556ca88c670 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:19.956341 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b51f64-79ac-4b45-bf67-3556ca88c670 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:19.957678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 712/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:19.963110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d49f9-2a77-47de-a390-c730f893a47f None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:19.963506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d49f9-2a77-47de-a390-c730f893a47f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:19.963664 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d49f9-2a77-47de-a390-c730f893a47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:20.013845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:20.031464 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-046d49f9-2a77-47de-a390-c730f893a47f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:20.037681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-046d49f9-2a77-47de-a390-c730f893a47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:20.127209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 736/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:20.416354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b51f64-79ac-4b45-bf67-3556ca88c670 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:20.417937 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-82b51f64-79ac-4b45-bf67-3556ca88c670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:20.519134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b51f64-79ac-4b45-bf67-3556ca88c670 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:20.558316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:20.591649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f931cdd-8d4f-4fd1-9b9f-3054a0c53e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:20.592333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f931cdd-8d4f-4fd1-9b9f-3054a0c53e6f service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:20.592788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f931cdd-8d4f-4fd1-9b9f-3054a0c53e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:20.593101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f931cdd-8d4f-4fd1-9b9f-3054a0c53e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:20.595210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7f931cdd-8d4f-4fd1-9b9f-3054a0c53e6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:20.705517 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f931cdd-8d4f-4fd1-9b9f-3054a0c53e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:20.706872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f931cdd-8d4f-4fd1-9b9f-3054a0c53e6f service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:20.726415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 713/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:23.097900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe34ca7-e746-4788-ba75-9292ddab274d None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:23.098467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe34ca7-e746-4788-ba75-9292ddab274d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:23.098467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe34ca7-e746-4788-ba75-9292ddab274d None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:23.099822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 737/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:23.106348 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f0b5d-58c2-4638-aff8-22b888e706ba None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:23.106754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f0b5d-58c2-4638-aff8-22b888e706ba None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:23.106896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f0b5d-58c2-4638-aff8-22b888e706ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:23.171388 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-db7f0b5d-58c2-4638-aff8-22b888e706ba None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:23.172573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-db7f0b5d-58c2-4638-aff8-22b888e706ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:23.193202 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:23.268392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 714/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:23.551294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe34ca7-e746-4788-ba75-9292ddab274d None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:23.552849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1fe34ca7-e746-4788-ba75-9292ddab274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:23.656760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe34ca7-e746-4788-ba75-9292ddab274d None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:23.675739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:23.729417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e055cb7-b7a5-4b31-9538-73ce2987884c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:23.729978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e055cb7-b7a5-4b31-9538-73ce2987884c service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:23.730258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e055cb7-b7a5-4b31-9538-73ce2987884c service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:23.730528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e055cb7-b7a5-4b31-9538-73ce2987884c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:23.732145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-3e055cb7-b7a5-4b31-9538-73ce2987884c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:23.838994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e055cb7-b7a5-4b31-9538-73ce2987884c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:23.840475 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e055cb7-b7a5-4b31-9538-73ce2987884c service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:23.858134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 738/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:26.084875 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f6e1d-da83-4fd1-8a4c-805306e955d3 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:26.085426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f6e1d-da83-4fd1-8a4c-805306e955d3 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:26.085426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f6e1d-da83-4fd1-8a4c-805306e955d3 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:26.087337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 715/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:26.092547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc975ce-682d-4c3c-9821-3ae3daccce56 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:26.092948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc975ce-682d-4c3c-9821-3ae3daccce56 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:26.093092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc975ce-682d-4c3c-9821-3ae3daccce56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:26.155065 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:26.157616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-fcc975ce-682d-4c3c-9821-3ae3daccce56 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:26.158801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fcc975ce-682d-4c3c-9821-3ae3daccce56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:26.256175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 739/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:26.539385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f6e1d-da83-4fd1-8a4c-805306e955d3 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:26.541003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-546f6e1d-da83-4fd1-8a4c-805306e955d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:26.645024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f6e1d-da83-4fd1-8a4c-805306e955d3 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:26.694011 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:26.718749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2137887-b0b3-45ac-97f4-fbffea754d42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:26.719349 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2137887-b0b3-45ac-97f4-fbffea754d42 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:26.719651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2137887-b0b3-45ac-97f4-fbffea754d42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:26.719950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2137887-b0b3-45ac-97f4-fbffea754d42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:26.721529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b2137887-b0b3-45ac-97f4-fbffea754d42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:26.829825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2137887-b0b3-45ac-97f4-fbffea754d42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:26.831199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2137887-b0b3-45ac-97f4-fbffea754d42 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:26.849498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 716/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:29.025746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6b0ff-460e-4208-bf97-42d7ce2abc89 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:29.026244 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6b0ff-460e-4208-bf97-42d7ce2abc89 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:29.026352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6b0ff-460e-4208-bf97-42d7ce2abc89 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:29.027639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 740/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:29.032787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde1e5-b773-40fb-84ea-9e00083c445d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:29.033230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde1e5-b773-40fb-84ea-9e00083c445d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:29.033342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fde1e5-b773-40fb-84ea-9e00083c445d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:29.099023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-21fde1e5-b773-40fb-84ea-9e00083c445d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:29.100175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-21fde1e5-b773-40fb-84ea-9e00083c445d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:29.128584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:29.195086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 717/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:29.481132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6b0ff-460e-4208-bf97-42d7ce2abc89 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:29.482646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-20b6b0ff-460e-4208-bf97-42d7ce2abc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:29.584675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b6b0ff-460e-4208-bf97-42d7ce2abc89 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:29.613376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:29.665056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263b950-e42e-422c-913e-683fe4b28b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:29.665659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b950-e42e-422c-913e-683fe4b28b47 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:29.665935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b950-e42e-422c-913e-683fe4b28b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:29.666205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263b950-e42e-422c-913e-683fe4b28b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:29.667794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5263b950-e42e-422c-913e-683fe4b28b47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:29.774938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263b950-e42e-422c-913e-683fe4b28b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:29.776330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263b950-e42e-422c-913e-683fe4b28b47 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:29.795204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 741/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:32.061066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa1a3e-70fb-4491-bc60-6fac7bebbbd7 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:32.061549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa1a3e-70fb-4491-bc60-6fac7bebbbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:32.061549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa1a3e-70fb-4491-bc60-6fac7bebbbd7 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:32.062940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 718/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:32.068310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd5b5-8558-4093-b70c-c7d0a866de46 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:32.068737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd5b5-8558-4093-b70c-c7d0a866de46 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:32.068868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd5b5-8558-4093-b70c-c7d0a866de46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:32.133968 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8d0dd5b5-8558-4093-b70c-c7d0a866de46 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:32.135166 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8d0dd5b5-8558-4093-b70c-c7d0a866de46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:32.143304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:32.230340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 742/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:32.512451 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa1a3e-70fb-4491-bc60-6fac7bebbbd7 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:32.513976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b2fa1a3e-70fb-4491-bc60-6fac7bebbbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:32.620813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa1a3e-70fb-4491-bc60-6fac7bebbbd7 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:32.676069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:32.694553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e7b60-45d8-4c32-86ac-8a58dd4d0c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:32.695154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7b60-45d8-4c32-86ac-8a58dd4d0c89 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:32.695978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7b60-45d8-4c32-86ac-8a58dd4d0c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:32.696251 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e7b60-45d8-4c32-86ac-8a58dd4d0c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:32.697834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-352e7b60-45d8-4c32-86ac-8a58dd4d0c89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:32.806645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e7b60-45d8-4c32-86ac-8a58dd4d0c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:32.808052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e7b60-45d8-4c32-86ac-8a58dd4d0c89 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:32.826380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 719/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:35.016273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb392f3-2c24-4d5a-be70-60c050fac175 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:35.016699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb392f3-2c24-4d5a-be70-60c050fac175 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:35.016834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb392f3-2c24-4d5a-be70-60c050fac175 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:35.018195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 743/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:35.024222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d74731-c541-4dcb-a8e3-f6141b1148fa None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:35.024648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d74731-c541-4dcb-a8e3-f6141b1148fa None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:35.024786 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d74731-c541-4dcb-a8e3-f6141b1148fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:35.090299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-54d74731-c541-4dcb-a8e3-f6141b1148fa None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:35.091656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-54d74731-c541-4dcb-a8e3-f6141b1148fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:35.129115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:35.186898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 720/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:16:35.468620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb392f3-2c24-4d5a-be70-60c050fac175 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:35.470136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ffb392f3-2c24-4d5a-be70-60c050fac175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:35.574210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb392f3-2c24-4d5a-be70-60c050fac175 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:35.611210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:35.647837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca9c34-f2c9-43a4-9f4b-77fe599fb196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:35.649023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca9c34-f2c9-43a4-9f4b-77fe599fb196 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:35.649336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca9c34-f2c9-43a4-9f4b-77fe599fb196 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:35.649615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca9c34-f2c9-43a4-9f4b-77fe599fb196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:35.651251 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-34ca9c34-f2c9-43a4-9f4b-77fe599fb196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:35.759202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca9c34-f2c9-43a4-9f4b-77fe599fb196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:35.760586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca9c34-f2c9-43a4-9f4b-77fe599fb196 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:35.779811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 744/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:37.946392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11bbb1-3e21-441f-90a2-bf0500dfd591 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:37.946950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11bbb1-3e21-441f-90a2-bf0500dfd591 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:37.946950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d11bbb1-3e21-441f-90a2-bf0500dfd591 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:37.948336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 721/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:37.953549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea2025-548e-4236-a456-d0a67a90c9de None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:37.953980 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea2025-548e-4236-a456-d0a67a90c9de None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:37.954121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea2025-548e-4236-a456-d0a67a90c9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:38.026624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-71ea2025-548e-4236-a456-d0a67a90c9de None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:38.027853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-71ea2025-548e-4236-a456-d0a67a90c9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:38.046157 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:38.123513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 745/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:38.410540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11bbb1-3e21-441f-90a2-bf0500dfd591 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:38.412141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6d11bbb1-3e21-441f-90a2-bf0500dfd591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:38.515561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d11bbb1-3e21-441f-90a2-bf0500dfd591 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:38.580997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:38.587455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf69cd8-8094-4078-936b-672b05f8f63a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:38.588081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf69cd8-8094-4078-936b-672b05f8f63a service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:38.588364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf69cd8-8094-4078-936b-672b05f8f63a service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:38.588637 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf69cd8-8094-4078-936b-672b05f8f63a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:38.590223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9bf69cd8-8094-4078-936b-672b05f8f63a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:38.697396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf69cd8-8094-4078-936b-672b05f8f63a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:38.698737 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf69cd8-8094-4078-936b-672b05f8f63a service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:38.716175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 722/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:40.879718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64678a7-4b30-40c1-9d65-cfb2903cfda7 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:40.880320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64678a7-4b30-40c1-9d65-cfb2903cfda7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:40.880320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64678a7-4b30-40c1-9d65-cfb2903cfda7 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:40.881644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 746/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:16:40.887549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9092b1f-8f46-41f8-90c4-96549d480c27 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:40.887997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9092b1f-8f46-41f8-90c4-96549d480c27 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:40.888151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9092b1f-8f46-41f8-90c4-96549d480c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:40.954120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-b9092b1f-8f46-41f8-90c4-96549d480c27 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:16:40.955314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b9092b1f-8f46-41f8-90c4-96549d480c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:41.004681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:41.064884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 723/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:16:41.363832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64678a7-4b30-40c1-9d65-cfb2903cfda7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:16:41.365436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b64678a7-4b30-40c1-9d65-cfb2903cfda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:41.468612 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64678a7-4b30-40c1-9d65-cfb2903cfda7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:16:41.515906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:41.541357 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3bc6e-55dd-4e59-8ef1-e19fb3f1632c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:16:41.541956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3bc6e-55dd-4e59-8ef1-e19fb3f1632c service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:16:41.542230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3bc6e-55dd-4e59-8ef1-e19fb3f1632c service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:16:41.542496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3bc6e-55dd-4e59-8ef1-e19fb3f1632c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:16:41.544102 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-79e3bc6e-55dd-4e59-8ef1-e19fb3f1632c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:16:41.652387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3bc6e-55dd-4e59-8ef1-e19fb3f1632c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:16:41.653822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3bc6e-55dd-4e59-8ef1-e19fb3f1632c service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:16:41.672491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 747/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:16:51.062989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=22 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:16:51.670205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=66 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:02.273120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022ac15-67ac-4df6-af61-d917b23b6b02 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:02.273673 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022ac15-67ac-4df6-af61-d917b23b6b02 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:02.273673 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022ac15-67ac-4df6-af61-d917b23b6b02 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:02.274940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 724/1471] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:17:02.280164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103d0da-49d6-442c-8171-62f7e5f336f0 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:02.280620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103d0da-49d6-442c-8171-62f7e5f336f0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:02.280725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1103d0da-49d6-442c-8171-62f7e5f336f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:02.346831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1103d0da-49d6-442c-8171-62f7e5f336f0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:17:02.348098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1103d0da-49d6-442c-8171-62f7e5f336f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:02.451087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 748/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:17:02.749233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a862799-72c5-4f23-95e3-c682742868a9 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:02.749515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a862799-72c5-4f23-95e3-c682742868a9 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:02.749792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a862799-72c5-4f23-95e3-c682742868a9 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:02.751154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 725/1473] 10.222.0.15 () {54 vars in 961 bytes} [Wed Sep 7 16:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:17:02.755963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfc887-4e54-43cb-aed3-17aa3c2323a0 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:02.756249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfc887-4e54-43cb-aed3-17aa3c2323a0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:02.756525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfc887-4e54-43cb-aed3-17aa3c2323a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:02.831367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-53bfc887-4e54-43cb-aed3-17aa3c2323a0 None None] MFA Rules not processed for user `0a5d7cf7534941cb857a4ab5867d978e`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:17:02.832593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-53bfc887-4e54-43cb-aed3-17aa3c2323a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:02.853657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:02.910034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 749/1474] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Sep 7 16:17:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:17:02.918529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a862799-72c5-4f23-95e3-c682742868a9 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:17:02.920068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9a862799-72c5-4f23-95e3-c682742868a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:03.025308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a862799-72c5-4f23-95e3-c682742868a9 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:17:03.097484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a890539c-13f6-4fb7-b28e-b5bbf96d099e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5d7cf7534941cb857a4ab5867d978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:17:03.098086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890539c-13f6-4fb7-b28e-b5bbf96d099e service glance] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:03.098368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890539c-13f6-4fb7-b28e-b5bbf96d099e service glance] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:03.098644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a890539c-13f6-4fb7-b28e-b5bbf96d099e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:03.100833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a890539c-13f6-4fb7-b28e-b5bbf96d099e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:03.153305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:03.208198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890539c-13f6-4fb7-b28e-b5bbf96d099e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:17:03.210264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a890539c-13f6-4fb7-b28e-b5bbf96d099e service glance] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:17:03.228412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 726/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:17:05.463278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35145e80-cfdf-4e2b-ad52-ed080d8721b7 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:05.463871 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35145e80-cfdf-4e2b-ad52-ed080d8721b7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:05.463871 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35145e80-cfdf-4e2b-ad52-ed080d8721b7 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:05.465188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 750/1476] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:17:05.470402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3319d4-b7b1-4d72-8919-86e333a80b38 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:05.470842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3319d4-b7b1-4d72-8919-86e333a80b38 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:05.470993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3319d4-b7b1-4d72-8919-86e333a80b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:05.544273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ed3319d4-b7b1-4d72-8919-86e333a80b38 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:17:05.545478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ed3319d4-b7b1-4d72-8919-86e333a80b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:05.641502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 727/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:17:05.913293 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35145e80-cfdf-4e2b-ad52-ed080d8721b7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:17:05.914822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-35145e80-cfdf-4e2b-ad52-ed080d8721b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:06.040805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35145e80-cfdf-4e2b-ad52-ed080d8721b7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:17:06.089753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:06.133769 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8990f505-e5c2-47b6-83dc-7ef2c92c0b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:17:06.134551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990f505-e5c2-47b6-83dc-7ef2c92c0b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:06.135030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990f505-e5c2-47b6-83dc-7ef2c92c0b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:06.135650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8990f505-e5c2-47b6-83dc-7ef2c92c0b49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:06.136385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990f505-e5c2-47b6-83dc-7ef2c92c0b49 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:17:06.138189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: WARNING py.warnings [None req-8990f505-e5c2-47b6-83dc-7ef2c92c0b49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:17:06.138189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: warnings.warn(msg) Sep 07 16:17:06.138189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: Sep 07 16:17:06.139326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8990f505-e5c2-47b6-83dc-7ef2c92c0b49 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:17:06.157712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 751/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Wed Sep 7 16:17:05 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:17:08.297423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702eea44-a51c-4b42-bbd1-aca28672b904 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:08.297945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702eea44-a51c-4b42-bbd1-aca28672b904 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:08.297945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702eea44-a51c-4b42-bbd1-aca28672b904 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:08.299245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 728/1479] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:17:08.304900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ffebd-f123-4a36-9de6-42dc6c5853a7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:08.305329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ffebd-f123-4a36-9de6-42dc6c5853a7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:08.305436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ffebd-f123-4a36-9de6-42dc6c5853a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:08.370795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6f5ffebd-f123-4a36-9de6-42dc6c5853a7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:17:08.372043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6f5ffebd-f123-4a36-9de6-42dc6c5853a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:08.467478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 752/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:17:08.759965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702eea44-a51c-4b42-bbd1-aca28672b904 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:17:08.761497 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-702eea44-a51c-4b42-bbd1-aca28672b904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:08.772040 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:08.907340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702eea44-a51c-4b42-bbd1-aca28672b904 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:17:08.985502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:09.018821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdf6f7-5f5c-418a-912b-11daad088937 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:17:09.019425 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdf6f7-5f5c-418a-912b-11daad088937 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:09.019702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdf6f7-5f5c-418a-912b-11daad088937 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:09.020002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdf6f7-5f5c-418a-912b-11daad088937 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:09.021617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-dbfdf6f7-5f5c-418a-912b-11daad088937 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:09.109316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdf6f7-5f5c-418a-912b-11daad088937 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:17:09.110885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdf6f7-5f5c-418a-912b-11daad088937 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:17:09.128932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 729/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:17:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:17:11.480827 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab55d75-2d24-4375-9970-c0ee5af0a536 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:11.481392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab55d75-2d24-4375-9970-c0ee5af0a536 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:11.481392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab55d75-2d24-4375-9970-c0ee5af0a536 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:11.482815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 753/1482] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:17:11.488354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3293e5-4d71-4501-8fb5-8dfc253a1ba0 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:11.488798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3293e5-4d71-4501-8fb5-8dfc253a1ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:11.488900 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3293e5-4d71-4501-8fb5-8dfc253a1ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:11.564250 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-8d3293e5-4d71-4501-8fb5-8dfc253a1ba0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:17:11.565467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8d3293e5-4d71-4501-8fb5-8dfc253a1ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:11.611653 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:11.661171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 730/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:17:11.959746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab55d75-2d24-4375-9970-c0ee5af0a536 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:17:11.961345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1ab55d75-2d24-4375-9970-c0ee5af0a536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:11.985131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:12.069718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab55d75-2d24-4375-9970-c0ee5af0a536 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:17:12.144352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2e416-d0a3-431b-8964-22fe02f6d930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:17:12.144957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2e416-d0a3-431b-8964-22fe02f6d930 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:12.145227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2e416-d0a3-431b-8964-22fe02f6d930 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:12.145494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2e416-d0a3-431b-8964-22fe02f6d930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:12.147603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-01d2e416-d0a3-431b-8964-22fe02f6d930 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:12.337690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:12.375441 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2e416-d0a3-431b-8964-22fe02f6d930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:17:12.377991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d2e416-d0a3-431b-8964-22fe02f6d930 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:17:12.406183 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 754/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:17:14.577390 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f7a2c-6820-4aab-a8ff-655861e64c39 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:14.577924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f7a2c-6820-4aab-a8ff-655861e64c39 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:14.577924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f7a2c-6820-4aab-a8ff-655861e64c39 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:14.579277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 731/1485] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:17:14.584495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde6a26-9190-41dc-9996-ca757784bfe7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:14.584899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde6a26-9190-41dc-9996-ca757784bfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:14.585035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dde6a26-9190-41dc-9996-ca757784bfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:14.648675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8dde6a26-9190-41dc-9996-ca757784bfe7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:17:14.649865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8dde6a26-9190-41dc-9996-ca757784bfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:14.744476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 755/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:17:15.029210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f7a2c-6820-4aab-a8ff-655861e64c39 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:17:15.030730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-544f7a2c-6820-4aab-a8ff-655861e64c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:15.133695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f7a2c-6820-4aab-a8ff-655861e64c39 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:17:15.203854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:15.205623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965788f-e04a-47c8-a6da-57b44a1d9a0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:17:15.206215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965788f-e04a-47c8-a6da-57b44a1d9a0a service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:15.206496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965788f-e04a-47c8-a6da-57b44a1d9a0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:15.206763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965788f-e04a-47c8-a6da-57b44a1d9a0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:15.208925 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0965788f-e04a-47c8-a6da-57b44a1d9a0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:15.315052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965788f-e04a-47c8-a6da-57b44a1d9a0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:17:15.316446 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965788f-e04a-47c8-a6da-57b44a1d9a0a service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:17:15.335174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 732/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:17:17.773407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11cfa6-db0d-44e0-8f37-932d4469ccda None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:17.773927 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11cfa6-db0d-44e0-8f37-932d4469ccda None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:17.773927 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11cfa6-db0d-44e0-8f37-932d4469ccda None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:17.775231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 756/1488] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:17:17.780581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7513391-c44d-49aa-9430-fe5943ff2812 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:17.781032 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7513391-c44d-49aa-9430-fe5943ff2812 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:17.781135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7513391-c44d-49aa-9430-fe5943ff2812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:17.846877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c7513391-c44d-49aa-9430-fe5943ff2812 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:17:17.848121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c7513391-c44d-49aa-9430-fe5943ff2812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:17.900614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:17.948380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 733/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:17:18.243069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11cfa6-db0d-44e0-8f37-932d4469ccda None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:17:18.247074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fd11cfa6-db0d-44e0-8f37-932d4469ccda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:18.281108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:18.347394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11cfa6-db0d-44e0-8f37-932d4469ccda None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:17:18.418728 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9164c-30c6-4039-8911-b940b5ee3f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:17:18.419321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9164c-30c6-4039-8911-b940b5ee3f0b service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:18.419601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9164c-30c6-4039-8911-b940b5ee3f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:18.419899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9164c-30c6-4039-8911-b940b5ee3f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:18.421463 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-83b9164c-30c6-4039-8911-b940b5ee3f0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:18.499313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:18.529508 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9164c-30c6-4039-8911-b940b5ee3f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:17:18.530932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9164c-30c6-4039-8911-b940b5ee3f0b service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:17:18.549098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 757/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:17:21.079562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31788908-03d4-4f46-be22-885f536ce3a5 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:21.080111 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31788908-03d4-4f46-be22-885f536ce3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:21.080111 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31788908-03d4-4f46-be22-885f536ce3a5 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:21.081464 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 734/1491] 10.222.0.15 () {54 vars in 927 bytes} [Wed Sep 7 16:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:17:21.086717 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede64e-b274-4e0c-9d11-6b95149e1ffb None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:21.087188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede64e-b274-4e0c-9d11-6b95149e1ffb None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:21.087280 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddede64e-b274-4e0c-9d11-6b95149e1ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:21.151231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ddede64e-b274-4e0c-9d11-6b95149e1ffb None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:17:21.152482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ddede64e-b274-4e0c-9d11-6b95149e1ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:21.247510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 758/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Wed Sep 7 16:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:17:21.540807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31788908-03d4-4f46-be22-885f536ce3a5 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:17:21.542359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-31788908-03d4-4f46-be22-885f536ce3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:21.647809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31788908-03d4-4f46-be22-885f536ce3a5 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:17:21.721813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60972e8-31da-4465-827f-48d952e2684b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:17:21.722415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60972e8-31da-4465-827f-48d952e2684b service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:17:21.722696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60972e8-31da-4465-827f-48d952e2684b service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:17:21.722967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60972e8-31da-4465-827f-48d952e2684b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:17:21.725280 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b60972e8-31da-4465-827f-48d952e2684b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:17:21.734702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:21.834484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60972e8-31da-4465-827f-48d952e2684b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:17:21.835859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60972e8-31da-4465-827f-48d952e2684b service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:17:21.854256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 735/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:17:31.246025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=82 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:17:31.851956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=50 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:43.384149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c6a13-c3c7-4b84-b363-dbd56487f191 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:43.384728 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c6a13-c3c7-4b84-b363-dbd56487f191 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:43.384728 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c6a13-c3c7-4b84-b363-dbd56487f191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:43.469566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6e8c6a13-c3c7-4b84-b363-dbd56487f191 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:18:43.471287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6e8c6a13-c3c7-4b84-b363-dbd56487f191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:43.565111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 759/1494] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:18:43.572896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90539a-b88d-4e77-b1bb-63d75ebf83eb None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:43.573402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90539a-b88d-4e77-b1bb-63d75ebf83eb None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:43.573518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90539a-b88d-4e77-b1bb-63d75ebf83eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:43.648750 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-df90539a-b88d-4e77-b1bb-63d75ebf83eb None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:18:43.649883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-df90539a-b88d-4e77-b1bb-63d75ebf83eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:43.744961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 736/1495] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:18:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:18:43.749593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c6a13-c3c7-4b84-b363-dbd56487f191 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:18:43.751355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6e8c6a13-c3c7-4b84-b363-dbd56487f191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:43.762492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c6a13-c3c7-4b84-b363-dbd56487f191 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:18:43.822905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:43.856177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d839ed-f137-411b-b029-c742799bdc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:18:43.856781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d839ed-f137-411b-b029-c742799bdc6f admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:43.857058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d839ed-f137-411b-b029-c742799bdc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:43.857331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d839ed-f137-411b-b029-c742799bdc6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:43.857967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d839ed-f137-411b-b029-c742799bdc6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:18:43.858267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d839ed-f137-411b-b029-c742799bdc6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:18:43.859285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: WARNING py.warnings [None req-87d839ed-f137-411b-b029-c742799bdc6f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:18:43.859285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: warnings.warn(msg) Sep 07 16:18:43.859285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: Sep 07 16:18:43.860197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d839ed-f137-411b-b029-c742799bdc6f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:18:43.871909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 760/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:18:43.880663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90539a-b88d-4e77-b1bb-63d75ebf83eb None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:18:43.889683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90539a-b88d-4e77-b1bb-63d75ebf83eb None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:18:43.957572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:43.995456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce55a9-51ed-47fc-b329-ac773b0a56fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:18:43.996091 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce55a9-51ed-47fc-b329-ac773b0a56fd admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:43.996373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce55a9-51ed-47fc-b329-ac773b0a56fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:43.996646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce55a9-51ed-47fc-b329-ac773b0a56fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:43.997288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce55a9-51ed-47fc-b329-ac773b0a56fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:18:43.998996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: WARNING py.warnings [None req-57ce55a9-51ed-47fc-b329-ac773b0a56fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:18:43.998996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: warnings.warn(msg) Sep 07 16:18:43.998996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: Sep 07 16:18:44.000231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce55a9-51ed-47fc-b329-ac773b0a56fd admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:18:44.098250 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 737/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:18:43 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:18:44.102586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d839ed-f137-411b-b029-c742799bdc6f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:18:44.104180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-87d839ed-f137-411b-b029-c742799bdc6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:44.138707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d839ed-f137-411b-b029-c742799bdc6f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:18:44.229791 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1eadcf-dd82-460f-87ff-aef3bad43a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:18:44.230326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eadcf-dd82-460f-87ff-aef3bad43a52 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:44.230601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eadcf-dd82-460f-87ff-aef3bad43a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:44.230869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eadcf-dd82-460f-87ff-aef3bad43a52 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:44.231492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1eadcf-dd82-460f-87ff-aef3bad43a52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:18:44.232515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: WARNING py.warnings [None req-1d1eadcf-dd82-460f-87ff-aef3bad43a52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:18:44.232515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: warnings.warn(msg) Sep 07 16:18:44.232515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: Sep 07 16:18:44.233730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1eadcf-dd82-460f-87ff-aef3bad43a52 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:18:44.336051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 761/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:18:44 2022] POST /identity/v3/users => generated 289 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:18:44.340321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce55a9-51ed-47fc-b329-ac773b0a56fd admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:18:44.341817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-57ce55a9-51ed-47fc-b329-ac773b0a56fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:44.402102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce55a9-51ed-47fc-b329-ac773b0a56fd admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:18:44.484393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:44.495179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44c05f-87c1-45ce-8abb-d2546765c903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:18:44.497539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44c05f-87c1-45ce-8abb-d2546765c903 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:44.497539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44c05f-87c1-45ce-8abb-d2546765c903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:44.497539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe44c05f-87c1-45ce-8abb-d2546765c903 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:44.497539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44c05f-87c1-45ce-8abb-d2546765c903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:18:44.498006 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: WARNING py.warnings [None req-fe44c05f-87c1-45ce-8abb-d2546765c903 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:18:44.498006 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: warnings.warn(msg) Sep 07 16:18:44.498006 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: Sep 07 16:18:44.498941 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe44c05f-87c1-45ce-8abb-d2546765c903 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:18:44.511591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 738/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:18:44.515687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1eadcf-dd82-460f-87ff-aef3bad43a52 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:18:44.519464 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1d1eadcf-dd82-460f-87ff-aef3bad43a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:44.528649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1eadcf-dd82-460f-87ff-aef3bad43a52 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:18:44.554764 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:44.623314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce36af-6574-47ea-9809-21a35b65e414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:18:44.623946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce36af-6574-47ea-9809-21a35b65e414 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:44.624226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce36af-6574-47ea-9809-21a35b65e414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:44.624498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce36af-6574-47ea-9809-21a35b65e414 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:44.625129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce36af-6574-47ea-9809-21a35b65e414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:18:44.626289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: WARNING py.warnings [None req-ebce36af-6574-47ea-9809-21a35b65e414 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:18:44.626289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: warnings.warn(msg) Sep 07 16:18:44.626289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: Sep 07 16:18:44.627176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce36af-6574-47ea-9809-21a35b65e414 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:18:44.639355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 762/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:18:44 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:18:44.643601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44c05f-87c1-45ce-8abb-d2546765c903 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:18:44.645237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-fe44c05f-87c1-45ce-8abb-d2546765c903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:44.678891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe44c05f-87c1-45ce-8abb-d2546765c903 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:18:44.773510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a9103-14d9-4060-b5c8-ef01c6656241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:18:44.774103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9103-14d9-4060-b5c8-ef01c6656241 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:44.774381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9103-14d9-4060-b5c8-ef01c6656241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:44.774654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a9103-14d9-4060-b5c8-ef01c6656241 admin admin] PATH_INFO: `/v3/projects/0a421a539872426d8a2953ad351e78de/users/0f1b61abc13e4e97b27c8569c59268b8/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:44.797672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a9103-14d9-4060-b5c8-ef01c6656241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a421a539872426d8a2953ad351e78de, user_id=0f1b61abc13e4e97b27c8569c59268b8, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:18:44.798720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: WARNING py.warnings [None req-087a9103-14d9-4060-b5c8-ef01c6656241 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:18:44.798720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: warnings.warn(msg) Sep 07 16:18:44.798720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: Sep 07 16:18:44.800685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a9103-14d9-4060-b5c8-ef01c6656241 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:18:44.882764 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 739/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:18:44 2022] PUT /identity/v3/projects/0a421a539872426d8a2953ad351e78de/users/0f1b61abc13e4e97b27c8569c59268b8/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:18:44.902698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e38127-f02f-4203-a63e-94191d9165d1 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:44.902985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e38127-f02f-4203-a63e-94191d9165d1 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:44.903259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e38127-f02f-4203-a63e-94191d9165d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:44.959524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-57e38127-f02f-4203-a63e-94191d9165d1 None None] MFA Rules not processed for user `0f1b61abc13e4e97b27c8569c59268b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:18:44.960754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-57e38127-f02f-4203-a63e-94191d9165d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:45.037089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 763/1502] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:18:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:18:45.056130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e013cf-c28c-4b49-b6b0-5ad3693a02fa None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:45.056619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e013cf-c28c-4b49-b6b0-5ad3693a02fa None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:45.056975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e013cf-c28c-4b49-b6b0-5ad3693a02fa None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:45.058318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 740/1503] 10.222.0.15 () {54 vars in 964 bytes} [Wed Sep 7 16:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:18:45.063477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42c4f7-eaeb-4ad7-b3b4-8ca7551d09e5 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:45.063785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42c4f7-eaeb-4ad7-b3b4-8ca7551d09e5 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:45.064060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d42c4f7-eaeb-4ad7-b3b4-8ca7551d09e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:45.095324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:45.140288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6d42c4f7-eaeb-4ad7-b3b4-8ca7551d09e5 None None] MFA Rules not processed for user `9f5f8dfbb9c44e788f110a4f53425344`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:18:45.142109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6d42c4f7-eaeb-4ad7-b3b4-8ca7551d09e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:45.216080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 764/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Wed Sep 7 16:18:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:18:45.224015 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e013cf-c28c-4b49-b6b0-5ad3693a02fa None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:18:45.225538 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-78e013cf-c28c-4b49-b6b0-5ad3693a02fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:45.275919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:45.329553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e013cf-c28c-4b49-b6b0-5ad3693a02fa None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:18:45.402770 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa34bea-5c55-4774-8c47-097e28589892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:18:45.403390 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa34bea-5c55-4774-8c47-097e28589892 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:45.403671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa34bea-5c55-4774-8c47-097e28589892 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:45.403973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa34bea-5c55-4774-8c47-097e28589892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:45.405642 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7fa34bea-5c55-4774-8c47-097e28589892 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:45.504658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa34bea-5c55-4774-8c47-097e28589892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:18:45.506081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa34bea-5c55-4774-8c47-097e28589892 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:18:45.509580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:45.523671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 741/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:18:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:18:45.569030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fafb9-a98a-40bc-818d-58e2bea7488f None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:45.569475 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fafb9-a98a-40bc-818d-58e2bea7488f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:45.569581 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fafb9-a98a-40bc-818d-58e2bea7488f None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:45.570831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 765/1506] 10.222.0.15 () {52 vars in 830 bytes} [Wed Sep 7 16:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:18:45.574454 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa34bea-5c55-4774-8c47-097e28589892 service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:18:45.575662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7fa34bea-5c55-4774-8c47-097e28589892 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:45.671980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa34bea-5c55-4774-8c47-097e28589892 service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:18:45.764490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:18:45.765140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:45.765419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:45.765706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] PATH_INFO: `/v3/users/0f1b61abc13e4e97b27c8569c59268b8` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:45.777689 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1b61abc13e4e97b27c8569c59268b8)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:18:45.778703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: WARNING py.warnings [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:18:45.778703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: warnings.warn(msg) Sep 07 16:18:45.778703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: Sep 07 16:18:45.779950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:18:45.789466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:46.008489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1b61abc13e4e97b27c8569c59268b8'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:18:46.084747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1b61abc13e4e97b27c8569c59268b8'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:18:46.085209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1b61abc13e4e97b27c8569c59268b8'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:18:46.085716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1b61abc13e4e97b27c8569c59268b8'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:18:46.100391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1b61abc13e4e97b27c8569c59268b8'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:18:46.128472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 742/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:18:45 2022] DELETE /identity/v3/users/0f1b61abc13e4e97b27c8569c59268b8 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:18:46.132823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e439-41aa-47a7-a203-d616a731090a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:46.133107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e439-41aa-47a7-a203-d616a731090a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:46.133378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4e439-41aa-47a7-a203-d616a731090a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:46.189011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-dfc4e439-41aa-47a7-a203-d616a731090a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:18:46.190201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-dfc4e439-41aa-47a7-a203-d616a731090a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:46.263454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:46.276217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 766/1508] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:18:46.286392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:18:46.289114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:46.435902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8fa6c-1a90-4c5f-8da4-7438e9dbfdc5 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:18:46.488747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:46.507846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced1f45-44e2-499c-92d6-749e34ffa7da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:18:46.508452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1f45-44e2-499c-92d6-749e34ffa7da service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:46.508726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1f45-44e2-499c-92d6-749e34ffa7da service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:46.508994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1f45-44e2-499c-92d6-749e34ffa7da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:46.510555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2ced1f45-44e2-499c-92d6-749e34ffa7da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:46.617014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced1f45-44e2-499c-92d6-749e34ffa7da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:18:46.618473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced1f45-44e2-499c-92d6-749e34ffa7da service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:18:46.636949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 743/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:18:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:18:47.099091 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4e439-41aa-47a7-a203-d616a731090a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:18:47.100337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-dfc4e439-41aa-47a7-a203-d616a731090a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:18:47.112265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4e439-41aa-47a7-a203-d616a731090a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:18:47.213439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:18:47.213980 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:18:47.214271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:18:47.214516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] PATH_INFO: `/v3/projects/0a421a539872426d8a2953ad351e78de` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:18:47.224987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a421a539872426d8a2953ad351e78de)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:18:47.226002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: WARNING py.warnings [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:18:47.226002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: warnings.warn(msg) Sep 07 16:18:47.226002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: Sep 07 16:18:47.227229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:18:47.356912 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a421a539872426d8a2953ad351e78de', 'request_id': 'req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:18:47.363475 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a421a539872426d8a2953ad351e78de', 'request_id': 'req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:18:47.421490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a421a539872426d8a2953ad351e78de', 'request_id': 'req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:18:47.452721 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] The token cache is being invalidate because project 0a421a539872426d8a2953ad351e78de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:18:47.453000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:18:47.453418 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f08e9adb-09a4-4bfd-a1f0-b7ddc0ee9555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:18:47.454843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 767/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:18:47 2022] DELETE /identity/v3/projects/0a421a539872426d8a2953ad351e78de => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:18:56.634696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=62 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:18:57.450780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:20:17.018929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65c3e2-6411-4656-ac66-55714783449e None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:17.019451 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65c3e2-6411-4656-ac66-55714783449e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:17.019451 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce65c3e2-6411-4656-ac66-55714783449e None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:17.020847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 744/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:20:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:20:17.025727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76fab1-9284-44dd-aa75-9ba6cda98839 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:17.026129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76fab1-9284-44dd-aa75-9ba6cda98839 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:17.026269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76fab1-9284-44dd-aa75-9ba6cda98839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:17.112483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-bd76fab1-9284-44dd-aa75-9ba6cda98839 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:20:17.113702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-bd76fab1-9284-44dd-aa75-9ba6cda98839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:17.218624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 768/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:20:17.371089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65c3e2-6411-4656-ac66-55714783449e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:20:17.373339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ce65c3e2-6411-4656-ac66-55714783449e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:17.492153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce65c3e2-6411-4656-ac66-55714783449e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:20:17.497678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef490363-e0f0-4cbc-b47e-b3f99773ff37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:20:17.498274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef490363-e0f0-4cbc-b47e-b3f99773ff37 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:17.498562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef490363-e0f0-4cbc-b47e-b3f99773ff37 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:17.498831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef490363-e0f0-4cbc-b47e-b3f99773ff37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:17.506651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef490363-e0f0-4cbc-b47e-b3f99773ff37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:20:17.508259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef490363-e0f0-4cbc-b47e-b3f99773ff37 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:20:17.517334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 745/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:20:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:20:18.398652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bc140-ac3b-4315-8f2a-7678845b2b17 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:18.399234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bc140-ac3b-4315-8f2a-7678845b2b17 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:18.399234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bc140-ac3b-4315-8f2a-7678845b2b17 None None] PATH_INFO: `/v3` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:18.400516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 769/1514] 10.222.0.50 () {54 vars in 925 bytes} [Wed Sep 7 16:20:18 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:20:18.406909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca2cad-a3ba-48b5-96fb-5c71b9bd8937 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:18.407353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca2cad-a3ba-48b5-96fb-5c71b9bd8937 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:18.407468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca2cad-a3ba-48b5-96fb-5c71b9bd8937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:18.467397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c6ca2cad-a3ba-48b5-96fb-5c71b9bd8937 None None] MFA Rules not processed for user `5fdfb0bf09db406ab5c09fbef887cbab`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:20:18.468631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c6ca2cad-a3ba-48b5-96fb-5c71b9bd8937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:18.537519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:20:18.542358 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 746/1515] 10.222.0.50 () {58 vars in 1015 bytes} [Wed Sep 7 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:20:18.595393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae58f-6122-425e-b463-e4e3d0555770 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:18.595674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae58f-6122-425e-b463-e4e3d0555770 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:18.595988 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae58f-6122-425e-b463-e4e3d0555770 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:18.597339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 770/1516] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:20:18.603230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9674-e961-4652-a9de-7e8ce8f6f888 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:18.603510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9674-e961-4652-a9de-7e8ce8f6f888 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:18.603802 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9674-e961-4652-a9de-7e8ce8f6f888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:18.625007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-16ea9674-e961-4652-a9de-7e8ce8f6f888 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:20:18.626194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-16ea9674-e961-4652-a9de-7e8ce8f6f888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:18.633911 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 747/1517] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:20:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:20:18.785434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ae58f-6122-425e-b463-e4e3d0555770 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:20:18.799947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ae58f-6122-425e-b463-e4e3d0555770 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:20:18.805521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cc872-14f6-4017-9079-849827516589 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:20:18.806104 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cc872-14f6-4017-9079-849827516589 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:18.806378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cc872-14f6-4017-9079-849827516589 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:18.806645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cc872-14f6-4017-9079-849827516589 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:18.815205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cc872-14f6-4017-9079-849827516589 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:20:18.816799 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cc872-14f6-4017-9079-849827516589 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:20:18.826046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 771/1518] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:20:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:20:27.495563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7333-7294-4679-87f1-a1096b4e47bd None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:27.496039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7333-7294-4679-87f1-a1096b4e47bd None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:27.496159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7333-7294-4679-87f1-a1096b4e47bd None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:27.498078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 748/1519] 10.222.0.15 () {54 vars in 961 bytes} [Wed Sep 7 16:20:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:20:27.503174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7b2a5-d80e-4cca-9938-9d5946ee5566 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:27.503628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7b2a5-d80e-4cca-9938-9d5946ee5566 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:27.503834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7b2a5-d80e-4cca-9938-9d5946ee5566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:27.522783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-08e7b2a5-d80e-4cca-9938-9d5946ee5566 None None] MFA Rules not processed for user `5fdfb0bf09db406ab5c09fbef887cbab`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:20:27.524101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-08e7b2a5-d80e-4cca-9938-9d5946ee5566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:27.532643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 772/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Sep 7 16:20:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:20:27.540292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce7333-7294-4679-87f1-a1096b4e47bd None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:20:27.554439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce7333-7294-4679-87f1-a1096b4e47bd None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:20:27.560021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0af8c-3d58-40d7-8bc1-bbdcb9d9da76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:20:27.560603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0af8c-3d58-40d7-8bc1-bbdcb9d9da76 service placement] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:27.560878 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0af8c-3d58-40d7-8bc1-bbdcb9d9da76 service placement] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:27.561150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0af8c-3d58-40d7-8bc1-bbdcb9d9da76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:27.569235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0af8c-3d58-40d7-8bc1-bbdcb9d9da76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:20:27.570805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0af8c-3d58-40d7-8bc1-bbdcb9d9da76 service placement] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:20:27.580260 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 749/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:20:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:20:30.276324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95250c91-29d2-41cc-85a2-9cc511fe1200 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:30.276847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95250c91-29d2-41cc-85a2-9cc511fe1200 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:30.276847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95250c91-29d2-41cc-85a2-9cc511fe1200 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:30.278646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 773/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:20:30.284925 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47854af8-c1c0-4916-b7cd-f2b4f66b4c36 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:30.285374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47854af8-c1c0-4916-b7cd-f2b4f66b4c36 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:30.285479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47854af8-c1c0-4916-b7cd-f2b4f66b4c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:30.305406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-47854af8-c1c0-4916-b7cd-f2b4f66b4c36 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:20:30.306600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-47854af8-c1c0-4916-b7cd-f2b4f66b4c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:30.314409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 750/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:20:30.472638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95250c91-29d2-41cc-85a2-9cc511fe1200 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:20:30.487965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95250c91-29d2-41cc-85a2-9cc511fe1200 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:20:30.493645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94450754-f6c1-4a5a-bbd1-6265a0bae52c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:20:30.494237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94450754-f6c1-4a5a-bbd1-6265a0bae52c service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:30.494514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94450754-f6c1-4a5a-bbd1-6265a0bae52c service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:30.494786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94450754-f6c1-4a5a-bbd1-6265a0bae52c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:30.503230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94450754-f6c1-4a5a-bbd1-6265a0bae52c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:20:30.504849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94450754-f6c1-4a5a-bbd1-6265a0bae52c service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:20:30.514708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 774/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:20:31.570722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24464d7d-305a-42dd-86f7-fce3bb41a2a0 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:31.571289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24464d7d-305a-42dd-86f7-fce3bb41a2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:31.571289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24464d7d-305a-42dd-86f7-fce3bb41a2a0 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:31.572621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 751/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 16:20:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:20:31.578534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca64f3-3333-4535-b4ae-ecdf83fbfb9c None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:31.578965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca64f3-3333-4535-b4ae-ecdf83fbfb9c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:31.579075 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca64f3-3333-4535-b4ae-ecdf83fbfb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:31.598972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1aca64f3-3333-4535-b4ae-ecdf83fbfb9c None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:20:31.600235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1aca64f3-3333-4535-b4ae-ecdf83fbfb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:31.608944 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 775/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 16:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:20:31.759401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24464d7d-305a-42dd-86f7-fce3bb41a2a0 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:20:31.774087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24464d7d-305a-42dd-86f7-fce3bb41a2a0 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:20:31.780445 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1691b6f-a59f-4381-8dee-086b712d220b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:20:31.781401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1691b6f-a59f-4381-8dee-086b712d220b service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:31.781670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1691b6f-a59f-4381-8dee-086b712d220b service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:31.781963 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1691b6f-a59f-4381-8dee-086b712d220b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:31.791319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1691b6f-a59f-4381-8dee-086b712d220b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:20:31.792966 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1691b6f-a59f-4381-8dee-086b712d220b service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:20:31.804339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 752/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:20:35.764863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd85c8-fe17-4f54-8a49-34f0515b35bd None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:35.765415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd85c8-fe17-4f54-8a49-34f0515b35bd None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:35.765415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd85c8-fe17-4f54-8a49-34f0515b35bd None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:35.766743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 776/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:20:35.772145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3ef04-069c-48c9-a0fa-46dd3e4234cf None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:35.772556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3ef04-069c-48c9-a0fa-46dd3e4234cf None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:35.772706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3ef04-069c-48c9-a0fa-46dd3e4234cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:35.792684 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-f5f3ef04-069c-48c9-a0fa-46dd3e4234cf None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:20:35.801502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f5f3ef04-069c-48c9-a0fa-46dd3e4234cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:35.801933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 753/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:20:36.094691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd85c8-fe17-4f54-8a49-34f0515b35bd None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:20:36.096480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-18bd85c8-fe17-4f54-8a49-34f0515b35bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:36.128744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:20:36.188266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd85c8-fe17-4f54-8a49-34f0515b35bd None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:20:36.191942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5420d7-45c5-4796-bfba-f0a8b6a1cf7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:20:36.192551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5420d7-45c5-4796-bfba-f0a8b6a1cf7e service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:36.192833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5420d7-45c5-4796-bfba-f0a8b6a1cf7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:36.193111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5420d7-45c5-4796-bfba-f0a8b6a1cf7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:36.202930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5420d7-45c5-4796-bfba-f0a8b6a1cf7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:20:36.204415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5420d7-45c5-4796-bfba-f0a8b6a1cf7e service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:20:36.214012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 777/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:20:38.375649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180dc1b-72f3-4fc3-937a-25009946c4ed None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:38.376203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180dc1b-72f3-4fc3-937a-25009946c4ed None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:38.376203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180dc1b-72f3-4fc3-937a-25009946c4ed None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:38.377627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 754/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:20:38.382915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e705d1-afc4-40ec-a850-28bdf8eb0705 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:38.383347 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e705d1-afc4-40ec-a850-28bdf8eb0705 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:38.383442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e705d1-afc4-40ec-a850-28bdf8eb0705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:38.403553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-77e705d1-afc4-40ec-a850-28bdf8eb0705 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:20:38.404795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-77e705d1-afc4-40ec-a850-28bdf8eb0705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:38.412597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 778/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:20:38.697870 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180dc1b-72f3-4fc3-937a-25009946c4ed None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:20:38.709236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180dc1b-72f3-4fc3-937a-25009946c4ed None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:20:38.712750 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f414f-02c5-41ec-a271-848b24a328bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:20:38.713334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f414f-02c5-41ec-a271-848b24a328bc service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:38.713606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f414f-02c5-41ec-a271-848b24a328bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:38.713871 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f414f-02c5-41ec-a271-848b24a328bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:38.723464 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f414f-02c5-41ec-a271-848b24a328bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:20:38.724972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f414f-02c5-41ec-a271-848b24a328bc service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:20:38.734403 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 755/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:20:42.522639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5cb4b-2324-495a-9a4b-1d58f3c2cf03 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:42.523227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5cb4b-2324-495a-9a4b-1d58f3c2cf03 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:42.523227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a5cb4b-2324-495a-9a4b-1d58f3c2cf03 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:42.524564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 779/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:20:42.529846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6dd66-983b-49da-9c92-740c15d8724d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:42.530243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6dd66-983b-49da-9c92-740c15d8724d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:42.530394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6dd66-983b-49da-9c92-740c15d8724d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:42.549954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-1cc6dd66-983b-49da-9c92-740c15d8724d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:20:42.551399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1cc6dd66-983b-49da-9c92-740c15d8724d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:42.559667 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 756/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:20:42.864000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5cb4b-2324-495a-9a4b-1d58f3c2cf03 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:20:42.865754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-53a5cb4b-2324-495a-9a4b-1d58f3c2cf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:42.943559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a5cb4b-2324-495a-9a4b-1d58f3c2cf03 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:20:42.947137 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f3b52-de15-4697-a31a-99dd754c3ba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:20:42.947736 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f3b52-de15-4697-a31a-99dd754c3ba4 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:42.948043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f3b52-de15-4697-a31a-99dd754c3ba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:42.948313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f3b52-de15-4697-a31a-99dd754c3ba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:42.957156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f3b52-de15-4697-a31a-99dd754c3ba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:20:42.958681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f3b52-de15-4697-a31a-99dd754c3ba4 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:20:42.968563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 780/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:20:45.093228 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d673aab-9755-4215-afb1-034139f393c3 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:45.093758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d673aab-9755-4215-afb1-034139f393c3 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:45.093758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d673aab-9755-4215-afb1-034139f393c3 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:45.095132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 757/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 16:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:20:45.100208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40b47d-ac18-4cb4-b0e1-b5f8be24a7df None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:45.100585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40b47d-ac18-4cb4-b0e1-b5f8be24a7df None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:45.100760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40b47d-ac18-4cb4-b0e1-b5f8be24a7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:45.119651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-9d40b47d-ac18-4cb4-b0e1-b5f8be24a7df None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:20:45.121388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9d40b47d-ac18-4cb4-b0e1-b5f8be24a7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:45.128848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 781/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 16:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:20:45.430944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d673aab-9755-4215-afb1-034139f393c3 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:20:45.432642 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8d673aab-9755-4215-afb1-034139f393c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:20:45.445520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d673aab-9755-4215-afb1-034139f393c3 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:20:45.449458 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c68d94-911c-4578-9f87-9d18372cb190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:20:45.450037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c68d94-911c-4578-9f87-9d18372cb190 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:20:45.450309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c68d94-911c-4578-9f87-9d18372cb190 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:20:45.450579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c68d94-911c-4578-9f87-9d18372cb190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:20:45.459156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c68d94-911c-4578-9f87-9d18372cb190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:20:45.460593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c68d94-911c-4578-9f87-9d18372cb190 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:20:45.470274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 758/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:20:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:20:55.114905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=63 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:20:55.466196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=37 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:13.034352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dfebd-895c-41d1-ae1a-ad2d3937038a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.035008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dfebd-895c-41d1-ae1a-ad2d3937038a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.035008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2dfebd-895c-41d1-ae1a-ad2d3937038a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.067434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6e2dfebd-895c-41d1-ae1a-ad2d3937038a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.068754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6e2dfebd-895c-41d1-ae1a-ad2d3937038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.081265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 782/1540] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.090893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63e2e-8e69-42be-a085-ee1042d581f9 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.100903 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63e2e-8e69-42be-a085-ee1042d581f9 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.100903 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63e2e-8e69-42be-a085-ee1042d581f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.119303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-d3c63e2e-8e69-42be-a085-ee1042d581f9 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.126078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d3c63e2e-8e69-42be-a085-ee1042d581f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.134665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 759/1541] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.139260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dfebd-895c-41d1-ae1a-ad2d3937038a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.157350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2dfebd-895c-41d1-ae1a-ad2d3937038a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:13.168588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3f55b-ff9f-4e53-8bf0-17d3b39b5519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:13.169401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3f55b-ff9f-4e53-8bf0-17d3b39b5519 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.169524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3f55b-ff9f-4e53-8bf0-17d3b39b5519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.169807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3f55b-ff9f-4e53-8bf0-17d3b39b5519 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.170477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3f55b-ff9f-4e53-8bf0-17d3b39b5519 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:13.170783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3f55b-ff9f-4e53-8bf0-17d3b39b5519 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:13.172793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3f55b-ff9f-4e53-8bf0-17d3b39b5519 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:13.193734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 783/1542] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:13.202304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c63e2e-8e69-42be-a085-ee1042d581f9 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.217662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c63e2e-8e69-42be-a085-ee1042d581f9 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:13.222840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a8eb8-bfd7-4af2-8ca6-fe8dae65a590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:13.223476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a8eb8-bfd7-4af2-8ca6-fe8dae65a590 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.223785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a8eb8-bfd7-4af2-8ca6-fe8dae65a590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.224058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a8eb8-bfd7-4af2-8ca6-fe8dae65a590 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.224703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a8eb8-bfd7-4af2-8ca6-fe8dae65a590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:13.227822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a8eb8-bfd7-4af2-8ca6-fe8dae65a590 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:13.302367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb220510-7b0b-4427-ba08-5494afb534cc None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.302732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb220510-7b0b-4427-ba08-5494afb534cc None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.302982 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb220510-7b0b-4427-ba08-5494afb534cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.331657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-bb220510-7b0b-4427-ba08-5494afb534cc None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.333929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-bb220510-7b0b-4427-ba08-5494afb534cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.346750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 784/1543] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:13.356570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa8991-e1f1-4071-a42e-dd52d099242e None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.356863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa8991-e1f1-4071-a42e-dd52d099242e None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.357126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfa8991-e1f1-4071-a42e-dd52d099242e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.364062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 760/1544] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:13.371628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a8eb8-bfd7-4af2-8ca6-fe8dae65a590 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.387107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-0dfa8991-e1f1-4071-a42e-dd52d099242e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.388361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0dfa8991-e1f1-4071-a42e-dd52d099242e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.475968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a8eb8-bfd7-4af2-8ca6-fe8dae65a590 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:13.487966 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded1f67-9a56-4d7e-8bdf-087cbf93eef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:13.488595 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded1f67-9a56-4d7e-8bdf-087cbf93eef7 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.488957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded1f67-9a56-4d7e-8bdf-087cbf93eef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.489170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded1f67-9a56-4d7e-8bdf-087cbf93eef7 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.489815 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded1f67-9a56-4d7e-8bdf-087cbf93eef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:13.491175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: WARNING py.warnings [None req-8ded1f67-9a56-4d7e-8bdf-087cbf93eef7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:22:13.491175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: warnings.warn(msg) Sep 07 16:22:13.491175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: Sep 07 16:22:13.492768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded1f67-9a56-4d7e-8bdf-087cbf93eef7 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:13.511415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 785/1545] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:13.513537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c63b63-a4af-4244-9bae-45a04f396b9c None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.513829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c63b63-a4af-4244-9bae-45a04f396b9c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.515467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c63b63-a4af-4244-9bae-45a04f396b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.537997 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-62c63b63-a4af-4244-9bae-45a04f396b9c None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.539218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-62c63b63-a4af-4244-9bae-45a04f396b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.547663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 786/1546] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.549331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3efcc3-fe93-49b2-ae66-a065437bdb34 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.549617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3efcc3-fe93-49b2-ae66-a065437bdb34 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.549898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3efcc3-fe93-49b2-ae66-a065437bdb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.570523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8f3efcc3-fe93-49b2-ae66-a065437bdb34 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.571732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8f3efcc3-fe93-49b2-ae66-a065437bdb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.579919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 787/1547] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.584337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96568-5313-4265-9a57-45e2253c7210 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.584337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96568-5313-4265-9a57-45e2253c7210 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.584337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d96568-5313-4265-9a57-45e2253c7210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.592836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 761/1548] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:13.593550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded1f67-9a56-4d7e-8bdf-087cbf93eef7 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.602154 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-66d96568-5313-4265-9a57-45e2253c7210 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.603126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded1f67-9a56-4d7e-8bdf-087cbf93eef7 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:13.603453 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-66d96568-5313-4265-9a57-45e2253c7210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.607679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373b078-8321-4e0d-9e15-b55ce5bee782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:13.608297 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373b078-8321-4e0d-9e15-b55ce5bee782 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.608572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373b078-8321-4e0d-9e15-b55ce5bee782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.608843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373b078-8321-4e0d-9e15-b55ce5bee782 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.609474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373b078-8321-4e0d-9e15-b55ce5bee782 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:13.609772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373b078-8321-4e0d-9e15-b55ce5bee782 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:13.610766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: WARNING py.warnings [None req-5373b078-8321-4e0d-9e15-b55ce5bee782 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:22:13.610766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: warnings.warn(msg) Sep 07 16:22:13.610766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: Sep 07 16:22:13.611354 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 788/1549] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.611649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373b078-8321-4e0d-9e15-b55ce5bee782 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:13.617191 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd8fdd-20d1-4cf9-ba9a-5a6f5fbba146 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.622099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd8fdd-20d1-4cf9-ba9a-5a6f5fbba146 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.622099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd8fdd-20d1-4cf9-ba9a-5a6f5fbba146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.624182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 762/1550] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:13.625526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e88411-0bf8-4239-8cba-56c1d21b8c3a None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.625821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e88411-0bf8-4239-8cba-56c1d21b8c3a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.626100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e88411-0bf8-4239-8cba-56c1d21b8c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.639539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-abbd8fdd-20d1-4cf9-ba9a-5a6f5fbba146 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.640818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-abbd8fdd-20d1-4cf9-ba9a-5a6f5fbba146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.645848 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-82e88411-0bf8-4239-8cba-56c1d21b8c3a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.648302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-82e88411-0bf8-4239-8cba-56c1d21b8c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.651909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 789/1551] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.651909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a32a1-cc87-43f1-a314-835befe35019 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.651909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a32a1-cc87-43f1-a314-835befe35019 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.651909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a32a1-cc87-43f1-a314-835befe35019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.658422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 763/1552] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.659699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e247b-a8b7-4b4f-a881-1894f8928a71 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.660033 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e247b-a8b7-4b4f-a881-1894f8928a71 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.660311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e247b-a8b7-4b4f-a881-1894f8928a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.671526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-408a32a1-cc87-43f1-a314-835befe35019 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.672778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-408a32a1-cc87-43f1-a314-835befe35019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.679693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-023e247b-a8b7-4b4f-a881-1894f8928a71 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.680894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-023e247b-a8b7-4b4f-a881-1894f8928a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.681032 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 790/1553] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.682084 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff146b9-6b69-4994-9065-8ba2e2252002 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.682369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff146b9-6b69-4994-9065-8ba2e2252002 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.682638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff146b9-6b69-4994-9065-8ba2e2252002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.691437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 764/1554] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.691612 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e247b-a8b7-4b4f-a881-1894f8928a71 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.701923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e247b-a8b7-4b4f-a881-1894f8928a71 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:13.706497 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54223d60-4a60-4d1a-92b5-3ed797f08682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:13.706725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-aff146b9-6b69-4994-9065-8ba2e2252002 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.707088 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54223d60-4a60-4d1a-92b5-3ed797f08682 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.707366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54223d60-4a60-4d1a-92b5-3ed797f08682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.707639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54223d60-4a60-4d1a-92b5-3ed797f08682 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.707955 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-aff146b9-6b69-4994-9065-8ba2e2252002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.708315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54223d60-4a60-4d1a-92b5-3ed797f08682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:13.710530 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54223d60-4a60-4d1a-92b5-3ed797f08682 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:13.716522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 791/1555] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.717546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321472d-0042-4fec-ba8b-d1447ba65f0d None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.717826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321472d-0042-4fec-ba8b-d1447ba65f0d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.718098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321472d-0042-4fec-ba8b-d1447ba65f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.723884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 765/1556] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:13.724692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54223d60-4a60-4d1a-92b5-3ed797f08682 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.734817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54223d60-4a60-4d1a-92b5-3ed797f08682 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:13.737509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b321472d-0042-4fec-ba8b-d1447ba65f0d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.738685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b321472d-0042-4fec-ba8b-d1447ba65f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.739438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e42407-4918-4d9d-bf9a-997fe7f951a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:13.740064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e42407-4918-4d9d-bf9a-997fe7f951a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.740342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e42407-4918-4d9d-bf9a-997fe7f951a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.740765 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e42407-4918-4d9d-bf9a-997fe7f951a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.741387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e42407-4918-4d9d-bf9a-997fe7f951a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:13.743272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e42407-4918-4d9d-bf9a-997fe7f951a3 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:13.746803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 792/1557] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.747819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdfa52-fb1d-481d-81ef-ab731a0adaed None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.748105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdfa52-fb1d-481d-81ef-ab731a0adaed None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.748376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdfa52-fb1d-481d-81ef-ab731a0adaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.767391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-5fbdfa52-fb1d-481d-81ef-ab731a0adaed None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.768654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5fbdfa52-fb1d-481d-81ef-ab731a0adaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.777079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 793/1558] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.778227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511c285-ba36-4961-90ad-f432aac22c25 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.778508 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511c285-ba36-4961-90ad-f432aac22c25 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.778779 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511c285-ba36-4961-90ad-f432aac22c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.797883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1511c285-ba36-4961-90ad-f432aac22c25 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.799300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1511c285-ba36-4961-90ad-f432aac22c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.807054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 794/1559] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.807831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511c285-ba36-4961-90ad-f432aac22c25 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.816816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511c285-ba36-4961-90ad-f432aac22c25 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:13.821898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3250e-4f2b-4c05-9bcc-7223907aa3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:13.822465 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3250e-4f2b-4c05-9bcc-7223907aa3ed admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.822740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3250e-4f2b-4c05-9bcc-7223907aa3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.823010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3250e-4f2b-4c05-9bcc-7223907aa3ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.823663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3250e-4f2b-4c05-9bcc-7223907aa3ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:13.823991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3250e-4f2b-4c05-9bcc-7223907aa3ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:13.825571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3250e-4f2b-4c05-9bcc-7223907aa3ed admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:13.837745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 795/1560] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:13.838915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbffc0-80d0-4953-a1ec-07ccddca8abc None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.839204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbffc0-80d0-4953-a1ec-07ccddca8abc None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.839529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbffc0-80d0-4953-a1ec-07ccddca8abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.854468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 766/1561] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:13.855548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e42407-4918-4d9d-bf9a-997fe7f951a3 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.858178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e2fbffc0-80d0-4953-a1ec-07ccddca8abc None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:13.859378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e2fbffc0-80d0-4953-a1ec-07ccddca8abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:13.909420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:13.919520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e42407-4918-4d9d-bf9a-997fe7f951a3 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:13.921697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 796/1562] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:13.922524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbffc0-80d0-4953-a1ec-07ccddca8abc None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.925052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58968a-888b-4b01-a9f5-7ff256eb90b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:13.925820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58968a-888b-4b01-a9f5-7ff256eb90b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.926108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58968a-888b-4b01-a9f5-7ff256eb90b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.926385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c58968a-888b-4b01-a9f5-7ff256eb90b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.927059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58968a-888b-4b01-a9f5-7ff256eb90b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:13.927383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58968a-888b-4b01-a9f5-7ff256eb90b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:13.927498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:13.929545 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c58968a-888b-4b01-a9f5-7ff256eb90b2 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:13.933364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbffc0-80d0-4953-a1ec-07ccddca8abc None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:13.938024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c919a-bedb-4109-92d3-63e5de7d23f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:13.938745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c919a-bedb-4109-92d3-63e5de7d23f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.939093 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c919a-bedb-4109-92d3-63e5de7d23f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.939988 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c919a-bedb-4109-92d3-63e5de7d23f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.940631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c919a-bedb-4109-92d3-63e5de7d23f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:13.943452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 767/1563] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:13.943590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c919a-bedb-4109-92d3-63e5de7d23f6 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:13.943707 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58968a-888b-4b01-a9f5-7ff256eb90b2 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.953001 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 797/1564] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:13.954068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c919a-bedb-4109-92d3-63e5de7d23f6 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.956669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c58968a-888b-4b01-a9f5-7ff256eb90b2 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:13.961727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168c308-f09f-4dca-9cdd-ea9a3f81242e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:13.962331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168c308-f09f-4dca-9cdd-ea9a3f81242e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.962615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168c308-f09f-4dca-9cdd-ea9a3f81242e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.962892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168c308-f09f-4dca-9cdd-ea9a3f81242e admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.963453 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c919a-bedb-4109-92d3-63e5de7d23f6 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:13.963578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168c308-f09f-4dca-9cdd-ea9a3f81242e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:13.963863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168c308-f09f-4dca-9cdd-ea9a3f81242e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:13.965588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168c308-f09f-4dca-9cdd-ea9a3f81242e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:13.968412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb16145-161e-485d-9d1c-541a1e61042e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:13.968993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb16145-161e-485d-9d1c-541a1e61042e admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:13.969274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb16145-161e-485d-9d1c-541a1e61042e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:13.969548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb16145-161e-485d-9d1c-541a1e61042e admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:13.970182 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb16145-161e-485d-9d1c-541a1e61042e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:13.970490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb16145-161e-485d-9d1c-541a1e61042e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:13.972090 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb16145-161e-485d-9d1c-541a1e61042e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:13.978837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 768/1565] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:13.979639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168c308-f09f-4dca-9cdd-ea9a3f81242e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.984482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 798/1566] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:13.985295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb16145-161e-485d-9d1c-541a1e61042e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:13.998951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb16145-161e-485d-9d1c-541a1e61042e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.001740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168c308-f09f-4dca-9cdd-ea9a3f81242e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.003456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f0fc7-2be6-4b57-afcb-637a389d6dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.004108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f0fc7-2be6-4b57-afcb-637a389d6dac admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.004389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f0fc7-2be6-4b57-afcb-637a389d6dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.004660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f0fc7-2be6-4b57-afcb-637a389d6dac admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.005286 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f0fc7-2be6-4b57-afcb-637a389d6dac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.006295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: WARNING py.warnings [None req-564f0fc7-2be6-4b57-afcb-637a389d6dac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:22:14.006295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: warnings.warn(msg) Sep 07 16:22:14.006295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: Sep 07 16:22:14.007396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e31d4a-0b68-482d-b17f-0d4ba2eac5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.008058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e31d4a-0b68-482d-b17f-0d4ba2eac5aa admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.008164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f0fc7-2be6-4b57-afcb-637a389d6dac admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.008341 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e31d4a-0b68-482d-b17f-0d4ba2eac5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.008607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e31d4a-0b68-482d-b17f-0d4ba2eac5aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.009253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e31d4a-0b68-482d-b17f-0d4ba2eac5aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:14.009681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e31d4a-0b68-482d-b17f-0d4ba2eac5aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.011269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e31d4a-0b68-482d-b17f-0d4ba2eac5aa admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.023152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 769/1567] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:14.028643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e31d4a-0b68-482d-b17f-0d4ba2eac5aa admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.038502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e31d4a-0b68-482d-b17f-0d4ba2eac5aa admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.043334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4b0f4-dc83-4c62-b8e8-22022341e6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.043968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4b0f4-dc83-4c62-b8e8-22022341e6a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.044251 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4b0f4-dc83-4c62-b8e8-22022341e6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.044518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4b0f4-dc83-4c62-b8e8-22022341e6a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.045125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4b0f4-dc83-4c62-b8e8-22022341e6a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.047097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4b0f4-dc83-4c62-b8e8-22022341e6a5 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.118120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 799/1568] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:13 2022] POST /identity/v3/projects => generated 389 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:14.119025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f0fc7-2be6-4b57-afcb-637a389d6dac admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.166935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 770/1569] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:14 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:14.168467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4b0f4-dc83-4c62-b8e8-22022341e6a5 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.179535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4b0f4-dc83-4c62-b8e8-22022341e6a5 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.180236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f0fc7-2be6-4b57-afcb-637a389d6dac admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.184650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77359f5-9920-4c14-914c-deea30ec2e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.184867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d3ae7-bff3-40b6-a7b3-3a3c7472cc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.185414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77359f5-9920-4c14-914c-deea30ec2e0f admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.185526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3ae7-bff3-40b6-a7b3-3a3c7472cc0d admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.185691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77359f5-9920-4c14-914c-deea30ec2e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.185791 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3ae7-bff3-40b6-a7b3-3a3c7472cc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.185963 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77359f5-9920-4c14-914c-deea30ec2e0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.186075 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d3ae7-bff3-40b6-a7b3-3a3c7472cc0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.186628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d3ae7-bff3-40b6-a7b3-3a3c7472cc0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:14.186742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77359f5-9920-4c14-914c-deea30ec2e0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.186931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d3ae7-bff3-40b6-a7b3-3a3c7472cc0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.188561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d3ae7-bff3-40b6-a7b3-3a3c7472cc0d admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.188687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77359f5-9920-4c14-914c-deea30ec2e0f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.200802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 800/1570] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:14.202170 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d3ae7-bff3-40b6-a7b3-3a3c7472cc0d admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.212127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d3ae7-bff3-40b6-a7b3-3a3c7472cc0d admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.217080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056210a-a58b-4698-a05f-56a037e2b3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.217721 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056210a-a58b-4698-a05f-56a037e2b3fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.218001 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056210a-a58b-4698-a05f-56a037e2b3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.218298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056210a-a58b-4698-a05f-56a037e2b3fc admin admin] PATH_INFO: `/v3/projects/2d0163a702614d6a946843060e6c8f24/users/95457e10373247639d89cc4fea72cac7/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.232729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056210a-a58b-4698-a05f-56a037e2b3fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d0163a702614d6a946843060e6c8f24, user_id=95457e10373247639d89cc4fea72cac7, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.233746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: WARNING py.warnings [None req-d056210a-a58b-4698-a05f-56a037e2b3fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:22:14.233746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: warnings.warn(msg) Sep 07 16:22:14.233746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: Sep 07 16:22:14.235750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056210a-a58b-4698-a05f-56a037e2b3fc admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.282596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 771/1571] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:14 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:14.283592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77359f5-9920-4c14-914c-deea30ec2e0f admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.310621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 801/1572] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:14 2022] PUT /identity/v3/projects/2d0163a702614d6a946843060e6c8f24/users/95457e10373247639d89cc4fea72cac7/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:14.311379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056210a-a58b-4698-a05f-56a037e2b3fc admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.347046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77359f5-9920-4c14-914c-deea30ec2e0f admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.373986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056210a-a58b-4698-a05f-56a037e2b3fc admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.378505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb8ad3-14b1-4cf1-86e0-f73ec6582157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.379108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb8ad3-14b1-4cf1-86e0-f73ec6582157 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.379384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb8ad3-14b1-4cf1-86e0-f73ec6582157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.379654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb8ad3-14b1-4cf1-86e0-f73ec6582157 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.380300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb8ad3-14b1-4cf1-86e0-f73ec6582157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.382217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb8ad3-14b1-4cf1-86e0-f73ec6582157 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.402869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cc1ed-7cfd-45da-bdd5-bb79d491148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.408344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cc1ed-7cfd-45da-bdd5-bb79d491148b admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.408344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cc1ed-7cfd-45da-bdd5-bb79d491148b admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.408344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864cc1ed-7cfd-45da-bdd5-bb79d491148b admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.408344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cc1ed-7cfd-45da-bdd5-bb79d491148b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.408344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864cc1ed-7cfd-45da-bdd5-bb79d491148b admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.463429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 802/1573] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:14 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:14.464260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb8ad3-14b1-4cf1-86e0-f73ec6582157 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.476138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:14.492368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 772/1574] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:14 2022] POST /identity/v3/projects => generated 363 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:14.493116 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cc1ed-7cfd-45da-bdd5-bb79d491148b admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.505706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:14.525899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb8ad3-14b1-4cf1-86e0-f73ec6582157 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.554557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864cc1ed-7cfd-45da-bdd5-bb79d491148b admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.559360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02ced0-1b27-460a-90a1-03125ce6c80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.560004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02ced0-1b27-460a-90a1-03125ce6c80d admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.560288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02ced0-1b27-460a-90a1-03125ce6c80d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.560563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02ced0-1b27-460a-90a1-03125ce6c80d admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.561189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02ced0-1b27-460a-90a1-03125ce6c80d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.563109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02ced0-1b27-460a-90a1-03125ce6c80d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.580995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65133a9a-fc69-4792-a4bf-6c4f16d84bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.581605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65133a9a-fc69-4792-a4bf-6c4f16d84bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.581887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65133a9a-fc69-4792-a4bf-6c4f16d84bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.582165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65133a9a-fc69-4792-a4bf-6c4f16d84bc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.582794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65133a9a-fc69-4792-a4bf-6c4f16d84bc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.585378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65133a9a-fc69-4792-a4bf-6c4f16d84bc6 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.652750 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 773/1575] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:14 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:14.653574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02ced0-1b27-460a-90a1-03125ce6c80d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.662798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02ced0-1b27-460a-90a1-03125ce6c80d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.667300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21c611-fb7c-46e7-8d3f-1a66da551d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.667905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21c611-fb7c-46e7-8d3f-1a66da551d78 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.668189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21c611-fb7c-46e7-8d3f-1a66da551d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.668473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec21c611-fb7c-46e7-8d3f-1a66da551d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.669090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21c611-fb7c-46e7-8d3f-1a66da551d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.670668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec21c611-fb7c-46e7-8d3f-1a66da551d78 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.681996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 803/1576] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:14 2022] POST /identity/v3/projects => generated 371 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:14.682874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65133a9a-fc69-4792-a4bf-6c4f16d84bc6 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.686168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 774/1577] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:14.686168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21c611-fb7c-46e7-8d3f-1a66da551d78 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.695510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec21c611-fb7c-46e7-8d3f-1a66da551d78 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.746220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65133a9a-fc69-4792-a4bf-6c4f16d84bc6 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.750775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ebb20-99c1-43a7-aa74-9c89f321ad38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.751380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ebb20-99c1-43a7-aa74-9c89f321ad38 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.751663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ebb20-99c1-43a7-aa74-9c89f321ad38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.751837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d621e-dedc-4ebe-91d9-71e96d1c5963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.751960 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ebb20-99c1-43a7-aa74-9c89f321ad38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.752407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d621e-dedc-4ebe-91d9-71e96d1c5963 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.752568 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ebb20-99c1-43a7-aa74-9c89f321ad38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.752677 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d621e-dedc-4ebe-91d9-71e96d1c5963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.752949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d621e-dedc-4ebe-91d9-71e96d1c5963 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.753601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d621e-dedc-4ebe-91d9-71e96d1c5963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.754559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ebb20-99c1-43a7-aa74-9c89f321ad38 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.755507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d621e-dedc-4ebe-91d9-71e96d1c5963 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.871049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 775/1578] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:14 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:14.872566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb796-2af5-44c1-8fd3-e62a097d0ddb None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.872853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb796-2af5-44c1-8fd3-e62a097d0ddb None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.873134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb796-2af5-44c1-8fd3-e62a097d0ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.882105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 804/1579] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:14 2022] POST /identity/v3/projects => generated 375 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:14.882419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ebb20-99c1-43a7-aa74-9c89f321ad38 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.902146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-69cdb796-2af5-44c1-8fd3-e62a097d0ddb None None] MFA Rules not processed for user `95457e10373247639d89cc4fea72cac7`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:14.903524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-69cdb796-2af5-44c1-8fd3-e62a097d0ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:14.912737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:14.946566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ebb20-99c1-43a7-aa74-9c89f321ad38 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.951314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca9bc0-33bc-4e23-8247-cc8468d6341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.952044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca9bc0-33bc-4e23-8247-cc8468d6341e admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.952321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca9bc0-33bc-4e23-8247-cc8468d6341e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.952591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca9bc0-33bc-4e23-8247-cc8468d6341e admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.953214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca9bc0-33bc-4e23-8247-cc8468d6341e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.955260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cca9bc0-33bc-4e23-8247-cc8468d6341e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:14.969544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 776/1580] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:14.970351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdb796-2af5-44c1-8fd3-e62a097d0ddb None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:14.980516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdb796-2af5-44c1-8fd3-e62a097d0ddb None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:14.985202 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70f363-e247-445c-b808-3f9776e15a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:14.985781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70f363-e247-445c-b808-3f9776e15a1a admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:14.986059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70f363-e247-445c-b808-3f9776e15a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:14.986324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70f363-e247-445c-b808-3f9776e15a1a admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:14.986939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70f363-e247-445c-b808-3f9776e15a1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:14.989271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe70f363-e247-445c-b808-3f9776e15a1a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.001270 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:15.018055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 805/1581] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:14 2022] POST /identity/v3/users => generated 283 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:15.018276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca9bc0-33bc-4e23-8247-cc8468d6341e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.027543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca9bc0-33bc-4e23-8247-cc8468d6341e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.032092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5a66d-3fc9-443a-bc8c-6cbdeb5e1c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.032663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5a66d-3fc9-443a-bc8c-6cbdeb5e1c52 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.032943 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5a66d-3fc9-443a-bc8c-6cbdeb5e1c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.033213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5a66d-3fc9-443a-bc8c-6cbdeb5e1c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.033833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5a66d-3fc9-443a-bc8c-6cbdeb5e1c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.035416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5a66d-3fc9-443a-bc8c-6cbdeb5e1c52 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.045703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 806/1582] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.046426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5a66d-3fc9-443a-bc8c-6cbdeb5e1c52 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.056698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5a66d-3fc9-443a-bc8c-6cbdeb5e1c52 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.063179 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01082e3d-1943-4e13-bf36-63e0d33e32fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.067271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 777/1583] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:14 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:15.067407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01082e3d-1943-4e13-bf36-63e0d33e32fd admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.067522 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70f363-e247-445c-b808-3f9776e15a1a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.067617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01082e3d-1943-4e13-bf36-63e0d33e32fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.067617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01082e3d-1943-4e13-bf36-63e0d33e32fd admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.067617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01082e3d-1943-4e13-bf36-63e0d33e32fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.067617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01082e3d-1943-4e13-bf36-63e0d33e32fd admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.073067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe70f363-e247-445c-b808-3f9776e15a1a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.078107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801343a-a127-4063-a7e1-e8177f975981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.078769 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801343a-a127-4063-a7e1-e8177f975981 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.079063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801343a-a127-4063-a7e1-e8177f975981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.079365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801343a-a127-4063-a7e1-e8177f975981 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.080063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801343a-a127-4063-a7e1-e8177f975981 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.081678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801343a-a127-4063-a7e1-e8177f975981 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.093012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 778/1584] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.093789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801343a-a127-4063-a7e1-e8177f975981 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.103484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801343a-a127-4063-a7e1-e8177f975981 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.108057 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530baf06-48b6-4891-a4e0-4773b7af4bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.108608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530baf06-48b6-4891-a4e0-4773b7af4bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.108885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530baf06-48b6-4891-a4e0-4773b7af4bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.109159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530baf06-48b6-4891-a4e0-4773b7af4bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.109792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530baf06-48b6-4891-a4e0-4773b7af4bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.111309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530baf06-48b6-4891-a4e0-4773b7af4bc5 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.121354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 779/1585] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.122069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530baf06-48b6-4891-a4e0-4773b7af4bc5 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.131254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530baf06-48b6-4891-a4e0-4773b7af4bc5 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.135913 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061e661-c521-4b34-9814-7dedc4fdaabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.138166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061e661-c521-4b34-9814-7dedc4fdaabe admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.138166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061e661-c521-4b34-9814-7dedc4fdaabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.138166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9061e661-c521-4b34-9814-7dedc4fdaabe admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.138166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061e661-c521-4b34-9814-7dedc4fdaabe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.139453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9061e661-c521-4b34-9814-7dedc4fdaabe admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.161123 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 807/1586] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:15 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:15.162351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b6d22-3432-4b83-8124-60b31c194253 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.162637 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b6d22-3432-4b83-8124-60b31c194253 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.163564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b6d22-3432-4b83-8124-60b31c194253 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.164975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 808/1587] 10.222.0.15 () {54 vars in 962 bytes} [Wed Sep 7 16:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:22:15.165472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b6d22-3432-4b83-8124-60b31c194253 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.177733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b6d22-3432-4b83-8124-60b31c194253 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.182766 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856dd618-ca67-4298-b902-6e3e997a53b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.183308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dd618-ca67-4298-b902-6e3e997a53b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.183581 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dd618-ca67-4298-b902-6e3e997a53b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.183880 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856dd618-ca67-4298-b902-6e3e997a53b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.184497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856dd618-ca67-4298-b902-6e3e997a53b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.186049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856dd618-ca67-4298-b902-6e3e997a53b7 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.196205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 809/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.198597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856dd618-ca67-4298-b902-6e3e997a53b7 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.206176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856dd618-ca67-4298-b902-6e3e997a53b7 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.210752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0635e1-6c2d-41d3-9d53-58c296c4f99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.211294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0635e1-6c2d-41d3-9d53-58c296c4f99e admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.211574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0635e1-6c2d-41d3-9d53-58c296c4f99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.211892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0635e1-6c2d-41d3-9d53-58c296c4f99e admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.212492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0635e1-6c2d-41d3-9d53-58c296c4f99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.214020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0635e1-6c2d-41d3-9d53-58c296c4f99e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.223508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 780/1589] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:15 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:15.223508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061e661-c521-4b34-9814-7dedc4fdaabe admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.231106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 810/1590] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.231106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0635e1-6c2d-41d3-9d53-58c296c4f99e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.233888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9061e661-c521-4b34-9814-7dedc4fdaabe admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.235706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0635e1-6c2d-41d3-9d53-58c296c4f99e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.238686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6335f7-ffd6-42cf-b8fb-1ff2a411b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.239315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6335f7-ffd6-42cf-b8fb-1ff2a411b39d admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.239558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6335f7-ffd6-42cf-b8fb-1ff2a411b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.239876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6335f7-ffd6-42cf-b8fb-1ff2a411b39d admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.240513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6335f7-ffd6-42cf-b8fb-1ff2a411b39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.240627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be5aba-75b4-4534-ac5f-0c440c43fe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.241852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5aba-75b4-4534-ac5f-0c440c43fe96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.242133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5aba-75b4-4534-ac5f-0c440c43fe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.242426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5aba-75b4-4534-ac5f-0c440c43fe96 admin admin] PATH_INFO: `/v3/projects/f6d21a3e70ba4789bfdfe42476d03a06/users/2eff82d212164719aacb10143d365259/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.242767 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6335f7-ffd6-42cf-b8fb-1ff2a411b39d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.261942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 781/1591] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.261942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6335f7-ffd6-42cf-b8fb-1ff2a411b39d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.262186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be5aba-75b4-4534-ac5f-0c440c43fe96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6d21a3e70ba4789bfdfe42476d03a06, user_id=2eff82d212164719aacb10143d365259, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.262186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be5aba-75b4-4534-ac5f-0c440c43fe96 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.263556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6335f7-ffd6-42cf-b8fb-1ff2a411b39d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.268555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08240a75-7b55-4e45-ad63-b286f4a8e38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.269149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08240a75-7b55-4e45-ad63-b286f4a8e38b admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.269429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08240a75-7b55-4e45-ad63-b286f4a8e38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.269701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08240a75-7b55-4e45-ad63-b286f4a8e38b admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.270332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08240a75-7b55-4e45-ad63-b286f4a8e38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.271988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08240a75-7b55-4e45-ad63-b286f4a8e38b admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.283241 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 782/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.283241 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08240a75-7b55-4e45-ad63-b286f4a8e38b admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.292627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08240a75-7b55-4e45-ad63-b286f4a8e38b admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.297256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70297f35-655a-4547-9884-dea6928397bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.297810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70297f35-655a-4547-9884-dea6928397bc admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.298099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70297f35-655a-4547-9884-dea6928397bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.298629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70297f35-655a-4547-9884-dea6928397bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.299237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70297f35-655a-4547-9884-dea6928397bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.300806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70297f35-655a-4547-9884-dea6928397bc admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.310912 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 783/1593] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.313707 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994ed7b-d7a0-4cca-ad9f-d2c528308140 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.313707 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994ed7b-d7a0-4cca-ad9f-d2c528308140 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.313707 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994ed7b-d7a0-4cca-ad9f-d2c528308140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.339019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 811/1594] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:15 2022] PUT /identity/v3/projects/f6d21a3e70ba4789bfdfe42476d03a06/users/2eff82d212164719aacb10143d365259/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:15.342484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be5aba-75b4-4534-ac5f-0c440c43fe96 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.349347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-1994ed7b-d7a0-4cca-ad9f-d2c528308140 None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:15.351108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1994ed7b-d7a0-4cca-ad9f-d2c528308140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:15.401177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be5aba-75b4-4534-ac5f-0c440c43fe96 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.405719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c96a9-f3e9-4e7a-8e1c-9c6db772e20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.406318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c96a9-f3e9-4e7a-8e1c-9c6db772e20a admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.406592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c96a9-f3e9-4e7a-8e1c-9c6db772e20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.406860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195c96a9-f3e9-4e7a-8e1c-9c6db772e20a admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.407513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c96a9-f3e9-4e7a-8e1c-9c6db772e20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.409123 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195c96a9-f3e9-4e7a-8e1c-9c6db772e20a admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.419307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 812/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.422832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c96a9-f3e9-4e7a-8e1c-9c6db772e20a admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.424250 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 784/1596] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Sep 7 16:22:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:15.425079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994ed7b-d7a0-4cca-ad9f-d2c528308140 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.432051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195c96a9-f3e9-4e7a-8e1c-9c6db772e20a admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.435284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994ed7b-d7a0-4cca-ad9f-d2c528308140 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.436785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5077487-69e7-48fd-ad04-86ac10e61cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.437413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5077487-69e7-48fd-ad04-86ac10e61cab admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.437688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5077487-69e7-48fd-ad04-86ac10e61cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.437957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5077487-69e7-48fd-ad04-86ac10e61cab admin admin] PATH_INFO: `/v3/projects/3d598cb3f17e407ab65dcdb8dbe3af8d/users/8430cbddaa1448f6ade09a082fa29b13/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.440070 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ae801-f1c2-4c20-a331-778f1712bf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.440658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ae801-f1c2-4c20-a331-778f1712bf8c admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.440942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ae801-f1c2-4c20-a331-778f1712bf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.441213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ae801-f1c2-4c20-a331-778f1712bf8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.441852 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ae801-f1c2-4c20-a331-778f1712bf8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.443446 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ae801-f1c2-4c20-a331-778f1712bf8c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.452384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5077487-69e7-48fd-ad04-86ac10e61cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d598cb3f17e407ab65dcdb8dbe3af8d, user_id=8430cbddaa1448f6ade09a082fa29b13, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.453435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 785/1597] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.454289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ae801-f1c2-4c20-a331-778f1712bf8c admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.455659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5077487-69e7-48fd-ad04-86ac10e61cab admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.464244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ae801-f1c2-4c20-a331-778f1712bf8c admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.468801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63ddd1-3405-4217-8315-c077c5df9bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.469378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63ddd1-3405-4217-8315-c077c5df9bef admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.469654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63ddd1-3405-4217-8315-c077c5df9bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.469924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63ddd1-3405-4217-8315-c077c5df9bef admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.470543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63ddd1-3405-4217-8315-c077c5df9bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.472133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63ddd1-3405-4217-8315-c077c5df9bef admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.482568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 786/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.483432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63ddd1-3405-4217-8315-c077c5df9bef admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.492726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63ddd1-3405-4217-8315-c077c5df9bef admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.497319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df53634-807f-4512-bf42-bd0dedcdc1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.497914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df53634-807f-4512-bf42-bd0dedcdc1c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.498193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df53634-807f-4512-bf42-bd0dedcdc1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.498467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df53634-807f-4512-bf42-bd0dedcdc1c9 admin admin] PATH_INFO: `/v3/projects/c2201e585106491d9551cd62ec095b45/users/9d617e7c7bdd4bcfb342c7b6ff223bb7/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.512614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df53634-807f-4512-bf42-bd0dedcdc1c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2201e585106491d9551cd62ec095b45, user_id=9d617e7c7bdd4bcfb342c7b6ff223bb7, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.515147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df53634-807f-4512-bf42-bd0dedcdc1c9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.584076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 813/1599] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:15 2022] PUT /identity/v3/projects/3d598cb3f17e407ab65dcdb8dbe3af8d/users/8430cbddaa1448f6ade09a082fa29b13/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:15.585551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5077487-69e7-48fd-ad04-86ac10e61cab admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.602990 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:15.612939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 787/1600] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:15 2022] PUT /identity/v3/projects/c2201e585106491d9551cd62ec095b45/users/9d617e7c7bdd4bcfb342c7b6ff223bb7/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:15.623718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441b9c-089e-45bc-8334-6ac5aa0e9178 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.623718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441b9c-089e-45bc-8334-6ac5aa0e9178 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.623718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53441b9c-089e-45bc-8334-6ac5aa0e9178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.639718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-53441b9c-089e-45bc-8334-6ac5aa0e9178 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:15.640950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-53441b9c-089e-45bc-8334-6ac5aa0e9178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:15.646091 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5077487-69e7-48fd-ad04-86ac10e61cab admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.650487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:15.701691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 788/1601] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:15.702320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd770762-48df-46ea-a1a1-e1798d9a2552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.702547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53441b9c-089e-45bc-8334-6ac5aa0e9178 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.702970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd770762-48df-46ea-a1a1-e1798d9a2552 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.703303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd770762-48df-46ea-a1a1-e1798d9a2552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.703581 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd770762-48df-46ea-a1a1-e1798d9a2552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.704249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd770762-48df-46ea-a1a1-e1798d9a2552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.706060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd770762-48df-46ea-a1a1-e1798d9a2552 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.714136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53441b9c-089e-45bc-8334-6ac5aa0e9178 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.718185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 814/1602] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.718938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd770762-48df-46ea-a1a1-e1798d9a2552 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.719735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ae723-52fd-406b-9c0b-23d449761da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.720436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ae723-52fd-406b-9c0b-23d449761da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.720694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ae723-52fd-406b-9c0b-23d449761da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.721188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ae723-52fd-406b-9c0b-23d449761da9 admin admin] PATH_INFO: `/v3/projects/9b4946c885204c5592a96bbeefde3ff1/users/0fc42e0e1c614ac3a953608eee0e6f08/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.736452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ae723-52fd-406b-9c0b-23d449761da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b4946c885204c5592a96bbeefde3ff1, user_id=0fc42e0e1c614ac3a953608eee0e6f08, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.739096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ae723-52fd-406b-9c0b-23d449761da9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.790044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd770762-48df-46ea-a1a1-e1798d9a2552 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.795420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04412784-5b67-47b0-9bd2-eebea663ccc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.796049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04412784-5b67-47b0-9bd2-eebea663ccc7 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.796325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04412784-5b67-47b0-9bd2-eebea663ccc7 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.796592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04412784-5b67-47b0-9bd2-eebea663ccc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.829754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 789/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:15 2022] PUT /identity/v3/projects/9b4946c885204c5592a96bbeefde3ff1/users/0fc42e0e1c614ac3a953608eee0e6f08/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:15.830678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ae723-52fd-406b-9c0b-23d449761da9 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.861603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:15.864036 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04412784-5b67-47b0-9bd2-eebea663ccc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.865627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04412784-5b67-47b0-9bd2-eebea663ccc7 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.874911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 815/1604] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:15.875540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04412784-5b67-47b0-9bd2-eebea663ccc7 service nova] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:15.887088 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04412784-5b67-47b0-9bd2-eebea663ccc7 service nova] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.892665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19210cd4-061e-4dbc-b1b5-715f96e31e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.893430 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19210cd4-061e-4dbc-b1b5-715f96e31e25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.893718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19210cd4-061e-4dbc-b1b5-715f96e31e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.894001 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19210cd4-061e-4dbc-b1b5-715f96e31e25 admin admin] PATH_INFO: `/v3/projects/b314ea662c5148abb80ea87a3e77cd19/users/5df7cf8384e04783a09b571e980b25ea/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.894186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ae723-52fd-406b-9c0b-23d449761da9 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:15.899480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a634a-0dd3-4c59-8db7-9ba6b2ab8b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:15.900133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a634a-0dd3-4c59-8db7-9ba6b2ab8b0e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.900416 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a634a-0dd3-4c59-8db7-9ba6b2ab8b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.900808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a634a-0dd3-4c59-8db7-9ba6b2ab8b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.901456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a634a-0dd3-4c59-8db7-9ba6b2ab8b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.903204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a634a-0dd3-4c59-8db7-9ba6b2ab8b0e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.913376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19210cd4-061e-4dbc-b1b5-715f96e31e25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b314ea662c5148abb80ea87a3e77cd19, user_id=5df7cf8384e04783a09b571e980b25ea, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:15.914843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 790/1605] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:15.916278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05222fde-6f37-43ca-821b-46f0d3cfab9d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.916513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19210cd4-061e-4dbc-b1b5-715f96e31e25 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:15.916619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05222fde-6f37-43ca-821b-46f0d3cfab9d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.916837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05222fde-6f37-43ca-821b-46f0d3cfab9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.936271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-05222fde-6f37-43ca-821b-46f0d3cfab9d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:15.937474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-05222fde-6f37-43ca-821b-46f0d3cfab9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:15.945554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 791/1606] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:15.946786 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ba5ad-7022-467c-9e33-967ae6ae0cf0 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:15.947072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ba5ad-7022-467c-9e33-967ae6ae0cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:15.947374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ba5ad-7022-467c-9e33-967ae6ae0cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:15.975461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-cf6ba5ad-7022-467c-9e33-967ae6ae0cf0 None None] MFA Rules not processed for user `9d617e7c7bdd4bcfb342c7b6ff223bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:15.976700 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-cf6ba5ad-7022-467c-9e33-967ae6ae0cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:16.001128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 816/1607] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:15 2022] PUT /identity/v3/projects/b314ea662c5148abb80ea87a3e77cd19/users/5df7cf8384e04783a09b571e980b25ea/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:16.002333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19210cd4-061e-4dbc-b1b5-715f96e31e25 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:16.050146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 792/1608] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:16.051189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ba5ad-7022-467c-9e33-967ae6ae0cf0 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:16.061564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ba5ad-7022-467c-9e33-967ae6ae0cf0 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:16.066418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:16.079942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19210cd4-061e-4dbc-b1b5-715f96e31e25 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:16.083807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb8379-32d3-4c2e-9e91-4f900ad0dfcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:16.084415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb8379-32d3-4c2e-9e91-4f900ad0dfcf service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.085340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb8379-32d3-4c2e-9e91-4f900ad0dfcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.085681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb8379-32d3-4c2e-9e91-4f900ad0dfcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.131367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a2f5e-1819-4ada-ad16-b0b7c774cc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:16.132762 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a2f5e-1819-4ada-ad16-b0b7c774cc69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.133137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a2f5e-1819-4ada-ad16-b0b7c774cc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.133511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a2f5e-1819-4ada-ad16-b0b7c774cc69 admin admin] PATH_INFO: `/v3/projects/fce8cd8db6664823be80953567300846/users/157b62754cf44d348387a142ec8a13a5/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.149420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a2f5e-1819-4ada-ad16-b0b7c774cc69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fce8cd8db6664823be80953567300846, user_id=157b62754cf44d348387a142ec8a13a5, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:16.152146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88a2f5e-1819-4ada-ad16-b0b7c774cc69 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:16.157476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb8379-32d3-4c2e-9e91-4f900ad0dfcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:16.158917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb8379-32d3-4c2e-9e91-4f900ad0dfcf service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:16.168550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 817/1609] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:16.176087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67faf2-eb36-4f79-b739-dde23f5b39d9 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.176087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67faf2-eb36-4f79-b739-dde23f5b39d9 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.176087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67faf2-eb36-4f79-b739-dde23f5b39d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.207476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-db67faf2-eb36-4f79-b739-dde23f5b39d9 None None] MFA Rules not processed for user `0fc42e0e1c614ac3a953608eee0e6f08`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:16.208737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-db67faf2-eb36-4f79-b739-dde23f5b39d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:16.261796 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 793/1610] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:16 2022] PUT /identity/v3/projects/fce8cd8db6664823be80953567300846/users/157b62754cf44d348387a142ec8a13a5/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:16.263230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a2f5e-1819-4ada-ad16-b0b7c774cc69 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:16.285781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 818/1611] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:16.287505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67faf2-eb36-4f79-b739-dde23f5b39d9 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:16.289421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-db67faf2-eb36-4f79-b739-dde23f5b39d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:16.307382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67faf2-eb36-4f79-b739-dde23f5b39d9 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:16.323049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:16.345444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88a2f5e-1819-4ada-ad16-b0b7c774cc69 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:16.351056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2d86c-4fef-486e-8d03-28efc49019d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:16.351954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2d86c-4fef-486e-8d03-28efc49019d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.358888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2d86c-4fef-486e-8d03-28efc49019d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.358888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2d86c-4fef-486e-8d03-28efc49019d2 admin admin] PATH_INFO: `/v3/projects/7170bcb816ec44899c533c914d861d8f/users/22c867d61b384f9e8f692b053526f1b3/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.384214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2d86c-4fef-486e-8d03-28efc49019d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7170bcb816ec44899c533c914d861d8f, user_id=22c867d61b384f9e8f692b053526f1b3, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:16.388171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2d86c-4fef-486e-8d03-28efc49019d2 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:16.393259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3729200-9ce4-4c3e-9b94-ef7639fe04dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:16.393259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3729200-9ce4-4c3e-9b94-ef7639fe04dd service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.393259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3729200-9ce4-4c3e-9b94-ef7639fe04dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.393670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3729200-9ce4-4c3e-9b94-ef7639fe04dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.403875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3729200-9ce4-4c3e-9b94-ef7639fe04dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:16.405391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3729200-9ce4-4c3e-9b94-ef7639fe04dd service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:16.415934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 819/1612] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:16.417801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962e38b-8865-440b-8364-90a80e15db22 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.418616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962e38b-8865-440b-8364-90a80e15db22 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.419230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2962e38b-8865-440b-8364-90a80e15db22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.447805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-2962e38b-8865-440b-8364-90a80e15db22 None None] MFA Rules not processed for user `5df7cf8384e04783a09b571e980b25ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:16.449015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2962e38b-8865-440b-8364-90a80e15db22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:16.487327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 794/1613] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:16 2022] PUT /identity/v3/projects/7170bcb816ec44899c533c914d861d8f/users/22c867d61b384f9e8f692b053526f1b3/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:16.488335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2d86c-4fef-486e-8d03-28efc49019d2 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:16.525499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 820/1614] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:16.528013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e621b3-bd00-400d-a786-dd3fcb3a83be None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.528150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e621b3-bd00-400d-a786-dd3fcb3a83be None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.528377 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e621b3-bd00-400d-a786-dd3fcb3a83be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.584230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2d86c-4fef-486e-8d03-28efc49019d2 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:16.590457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b1e621b3-bd00-400d-a786-dd3fcb3a83be None None] MFA Rules not processed for user `157b62754cf44d348387a142ec8a13a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:16.590747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b482284d-796d-4d30-9a04-d1492f2fce38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:16.591323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482284d-796d-4d30-9a04-d1492f2fce38 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.591601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482284d-796d-4d30-9a04-d1492f2fce38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.591924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b482284d-796d-4d30-9a04-d1492f2fce38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.592569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482284d-796d-4d30-9a04-d1492f2fce38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:16.593460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b1e621b3-bd00-400d-a786-dd3fcb3a83be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:16.595735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b482284d-796d-4d30-9a04-d1492f2fce38 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:16.676558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 821/1615] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:16.677719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e621b3-bd00-400d-a786-dd3fcb3a83be None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:16.683375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:16.694492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 795/1616] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:16 2022] POST /identity/v3/projects => generated 371 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:16.695863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f7915-db11-46af-a77a-cf19351d8bd5 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.696002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e621b3-bd00-400d-a786-dd3fcb3a83be None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:16.696147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f7915-db11-46af-a77a-cf19351d8bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.696419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f7915-db11-46af-a77a-cf19351d8bd5 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.697788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 796/1617] 10.222.0.15 () {54 vars in 962 bytes} [Wed Sep 7 16:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:22:16.698795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783e8bc-65e9-4768-b6ca-a01d1b6a13a5 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.699080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783e8bc-65e9-4768-b6ca-a01d1b6a13a5 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.699466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783e8bc-65e9-4768-b6ca-a01d1b6a13a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.719136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-b783e8bc-65e9-4768-b6ca-a01d1b6a13a5 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:16.720384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b783e8bc-65e9-4768-b6ca-a01d1b6a13a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:16.727995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:16.757364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244fa28-8398-4576-bd44-ff8187622e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:16.757994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244fa28-8398-4576-bd44-ff8187622e96 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.758271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244fa28-8398-4576-bd44-ff8187622e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.758546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244fa28-8398-4576-bd44-ff8187622e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.759190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244fa28-8398-4576-bd44-ff8187622e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:16.761261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5244fa28-8398-4576-bd44-ff8187622e96 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:16.791180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 797/1618] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:16.792357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783e8bc-65e9-4768-b6ca-a01d1b6a13a5 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:16.858669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783e8bc-65e9-4768-b6ca-a01d1b6a13a5 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:16.863212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 822/1619] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:16 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:16.863212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244fa28-8398-4576-bd44-ff8187622e96 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:16.952180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244fa28-8398-4576-bd44-ff8187622e96 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:16.953275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5f783-3d5d-4cd9-ac5f-53c30661851b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:16.953986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f783-3d5d-4cd9-ac5f-53c30661851b service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.954278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f783-3d5d-4cd9-ac5f-53c30661851b service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.954601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5f783-3d5d-4cd9-ac5f-53c30661851b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:16.958553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723c5eae-ec20-4a1d-bb1b-55559a21c668 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:16.959167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c5eae-ec20-4a1d-bb1b-55559a21c668 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:16.960051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c5eae-ec20-4a1d-bb1b-55559a21c668 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:16.960366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723c5eae-ec20-4a1d-bb1b-55559a21c668 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.026842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5f783-3d5d-4cd9-ac5f-53c30661851b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.029592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5f783-3d5d-4cd9-ac5f-53c30661851b service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.042857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 798/1620] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:17.044235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5f783-3d5d-4cd9-ac5f-53c30661851b service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.044716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c5eae-ec20-4a1d-bb1b-55559a21c668 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.061958 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:17.062694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723c5eae-ec20-4a1d-bb1b-55559a21c668 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.104516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 823/1621] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:17.106797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78393a-8673-4177-8065-cb30a596b80b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.107323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78393a-8673-4177-8065-cb30a596b80b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.108146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c78393a-8673-4177-8065-cb30a596b80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.151201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-0c78393a-8673-4177-8065-cb30a596b80b None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:17.154096 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0c78393a-8673-4177-8065-cb30a596b80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:17.162124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5f783-3d5d-4cd9-ac5f-53c30661851b service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.169123 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 824/1622] 10.222.0.15 () {58 vars in 1059 bytes} [Wed Sep 7 16:22:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:17.170456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78393a-8673-4177-8065-cb30a596b80b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.179128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e44e7-5193-484d-8031-492888d8a5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.179874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e44e7-5193-484d-8031-492888d8a5ae admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.181351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e44e7-5193-484d-8031-492888d8a5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.181702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e44e7-5193-484d-8031-492888d8a5ae admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.183113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e44e7-5193-484d-8031-492888d8a5ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.186985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e44e7-5193-484d-8031-492888d8a5ae admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.285447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:17.293878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c78393a-8673-4177-8065-cb30a596b80b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.300927 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d7bda-9feb-4f92-9a90-b29c1be3f463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.301893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d7bda-9feb-4f92-9a90-b29c1be3f463 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.302418 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d7bda-9feb-4f92-9a90-b29c1be3f463 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.302879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d7bda-9feb-4f92-9a90-b29c1be3f463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.327859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d7bda-9feb-4f92-9a90-b29c1be3f463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.328051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 799/1623] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:17 2022] POST /identity/v3/users => generated 292 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:17.328051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e44e7-5193-484d-8031-492888d8a5ae admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.328974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0d7bda-9feb-4f92-9a90-b29c1be3f463 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.344712 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e44e7-5193-484d-8031-492888d8a5ae admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.349218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 825/1624] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:17.352319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a4505-4389-4401-a9df-604cd30c36f8 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.352803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a4505-4389-4401-a9df-604cd30c36f8 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.358989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a4505-4389-4401-a9df-604cd30c36f8 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.358989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 826/1625] 10.222.0.15 () {54 vars in 912 bytes} [Wed Sep 7 16:22:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:22:17.359741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11bf3-9335-41df-a4f4-c982588ee804 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.360531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11bf3-9335-41df-a4f4-c982588ee804 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.361232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf11bf3-9335-41df-a4f4-c982588ee804 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.362985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5960db4-bb9c-4650-a88b-f68b5d89fc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.363658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5960db4-bb9c-4650-a88b-f68b5d89fc56 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.363965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5960db4-bb9c-4650-a88b-f68b5d89fc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.364236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5960db4-bb9c-4650-a88b-f68b5d89fc56 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.364760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 827/1626] 10.222.0.15 () {54 vars in 912 bytes} [Wed Sep 7 16:22:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:22:17.366497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf11bf3-9335-41df-a4f4-c982588ee804 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.369680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5960db4-bb9c-4650-a88b-f68b5d89fc56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.377440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5960db4-bb9c-4650-a88b-f68b5d89fc56 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.395576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf11bf3-9335-41df-a4f4-c982588ee804 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.401177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ecd2d-c0ff-4d65-a257-4c767e3f9fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.401992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ecd2d-c0ff-4d65-a257-4c767e3f9fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.402128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ecd2d-c0ff-4d65-a257-4c767e3f9fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.402323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ecd2d-c0ff-4d65-a257-4c767e3f9fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.403678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ecd2d-c0ff-4d65-a257-4c767e3f9fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.404863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ecd2d-c0ff-4d65-a257-4c767e3f9fb0 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.420599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 828/1627] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:17.420599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52146f-1405-4060-b060-0f1652356cfd None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.420599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52146f-1405-4060-b060-0f1652356cfd None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.420599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52146f-1405-4060-b060-0f1652356cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.443674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1c52146f-1405-4060-b060-0f1652356cfd None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:17.445509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1c52146f-1405-4060-b060-0f1652356cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:17.456603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 829/1628] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:22:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:17.467340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bfa4e-2aea-407b-bd34-939902214141 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.467975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bfa4e-2aea-407b-bd34-939902214141 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.468854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bfa4e-2aea-407b-bd34-939902214141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.476359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 800/1629] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:17 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:17.477787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5960db4-bb9c-4650-a88b-f68b5d89fc56 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.490800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5960db4-bb9c-4650-a88b-f68b5d89fc56 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.496048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3efe6-7d35-4b18-b41b-cbdb03bed272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.496845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3efe6-7d35-4b18-b41b-cbdb03bed272 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.497607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3efe6-7d35-4b18-b41b-cbdb03bed272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.497607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3efe6-7d35-4b18-b41b-cbdb03bed272 admin admin] PATH_INFO: `/v3/projects/f248d78c873942cab2bb2c57369fe4db/users/b032ba8bfcc54f1d974ac32db61526ae/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.511163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-9e7bfa4e-2aea-407b-bd34-939902214141 None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:17.513480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9e7bfa4e-2aea-407b-bd34-939902214141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:17.516329 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3efe6-7d35-4b18-b41b-cbdb03bed272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f248d78c873942cab2bb2c57369fe4db, user_id=b032ba8bfcc54f1d974ac32db61526ae, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.519252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3efe6-7d35-4b18-b41b-cbdb03bed272 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.525948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 830/1630] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:22:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:17.533499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bfa4e-2aea-407b-bd34-939902214141 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.544365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bfa4e-2aea-407b-bd34-939902214141 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.548512 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f610ef-590f-40a3-a842-65d8bf7e7614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.549517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f610ef-590f-40a3-a842-65d8bf7e7614 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.549517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f610ef-590f-40a3-a842-65d8bf7e7614 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.549716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f610ef-590f-40a3-a842-65d8bf7e7614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.560749 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f610ef-590f-40a3-a842-65d8bf7e7614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.562227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f610ef-590f-40a3-a842-65d8bf7e7614 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.574575 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 831/1631] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:17.575516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f610ef-590f-40a3-a842-65d8bf7e7614 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.588921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f610ef-590f-40a3-a842-65d8bf7e7614 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.594903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5ce0b-9da0-4871-bcc6-f63286675fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.595633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5ce0b-9da0-4871-bcc6-f63286675fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.595999 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5ce0b-9da0-4871-bcc6-f63286675fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.596363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5ce0b-9da0-4871-bcc6-f63286675fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.597696 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5ce0b-9da0-4871-bcc6-f63286675fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.599496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5ce0b-9da0-4871-bcc6-f63286675fc2 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.615389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 832/1632] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:17.617147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5ce0b-9da0-4871-bcc6-f63286675fc2 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.629930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5ce0b-9da0-4871-bcc6-f63286675fc2 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.637128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e61963-3e66-4b58-a670-98233abc84ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.637949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e61963-3e66-4b58-a670-98233abc84ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.638311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e61963-3e66-4b58-a670-98233abc84ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.638795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e61963-3e66-4b58-a670-98233abc84ff admin admin] PATH_INFO: `/v3/projects/ca5d9692a8c24d49925e1935ae2cd929/users/7fd31f1e4f3d440c9227eb69d9cd5ee3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.653806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 801/1633] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:17 2022] PUT /identity/v3/projects/f248d78c873942cab2bb2c57369fe4db/users/b032ba8bfcc54f1d974ac32db61526ae/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:17.654613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3efe6-7d35-4b18-b41b-cbdb03bed272 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.673412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e61963-3e66-4b58-a670-98233abc84ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5d9692a8c24d49925e1935ae2cd929, user_id=7fd31f1e4f3d440c9227eb69d9cd5ee3, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.676511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e61963-3e66-4b58-a670-98233abc84ff admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.751596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 833/1634] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:17 2022] PUT /identity/v3/projects/ca5d9692a8c24d49925e1935ae2cd929/users/7fd31f1e4f3d440c9227eb69d9cd5ee3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:17.752935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e61963-3e66-4b58-a670-98233abc84ff admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.768809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3efe6-7d35-4b18-b41b-cbdb03bed272 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.823904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e61963-3e66-4b58-a670-98233abc84ff admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.828156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc1e97-b820-446f-83ec-24ef574373a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.829162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1e97-b820-446f-83ec-24ef574373a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.829545 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1e97-b820-446f-83ec-24ef574373a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.829918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc1e97-b820-446f-83ec-24ef574373a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.843813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d84610-16aa-4375-9fe5-d7be978754ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.844799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d84610-16aa-4375-9fe5-d7be978754ee admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.845125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d84610-16aa-4375-9fe5-d7be978754ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.845389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d84610-16aa-4375-9fe5-d7be978754ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.846039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d84610-16aa-4375-9fe5-d7be978754ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.847845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d84610-16aa-4375-9fe5-d7be978754ee admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.871883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 802/1635] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:22:17.872744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d84610-16aa-4375-9fe5-d7be978754ee admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.881183 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:17.892317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d84610-16aa-4375-9fe5-d7be978754ee admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.897378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cddb4-026e-491a-8241-4416c826dcd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.898813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cddb4-026e-491a-8241-4416c826dcd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.898813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cddb4-026e-491a-8241-4416c826dcd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.899366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cddb4-026e-491a-8241-4416c826dcd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.900087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710cddb4-026e-491a-8241-4416c826dcd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.901702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710cddb4-026e-491a-8241-4416c826dcd8 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.929449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 803/1636] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:17.930378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cddb4-026e-491a-8241-4416c826dcd8 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.942808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc1e97-b820-446f-83ec-24ef574373a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.945381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc1e97-b820-446f-83ec-24ef574373a4 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.952855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cddb4-026e-491a-8241-4416c826dcd8 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.959295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 834/1637] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:17.960696 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc1e97-b820-446f-83ec-24ef574373a4 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:17.961811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385343c-c6c3-4aa8-b84a-d61538db0dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.962773 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385343c-c6c3-4aa8-b84a-d61538db0dae admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.963378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385343c-c6c3-4aa8-b84a-d61538db0dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.963710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385343c-c6c3-4aa8-b84a-d61538db0dae admin admin] PATH_INFO: `/v3/projects/f248d78c873942cab2bb2c57369fe4db/users/b032ba8bfcc54f1d974ac32db61526ae/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.966320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385343c-c6c3-4aa8-b84a-d61538db0dae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f248d78c873942cab2bb2c57369fe4db, user_id=b032ba8bfcc54f1d974ac32db61526ae, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.970909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385343c-c6c3-4aa8-b84a-d61538db0dae admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:17.975892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc1e97-b820-446f-83ec-24ef574373a4 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:17.981645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740e147-a34a-45d8-91ef-c398fb6e069d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:17.982563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740e147-a34a-45d8-91ef-c398fb6e069d admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:17.983217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740e147-a34a-45d8-91ef-c398fb6e069d admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:17.985715 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740e147-a34a-45d8-91ef-c398fb6e069d admin admin] PATH_INFO: `/v3/projects/ca5d9692a8c24d49925e1935ae2cd929/users/7fd31f1e4f3d440c9227eb69d9cd5ee3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:17.996446 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 804/1638] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:17 2022] PUT /identity/v3/projects/f248d78c873942cab2bb2c57369fe4db/users/b032ba8bfcc54f1d974ac32db61526ae/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:17.996644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740e147-a34a-45d8-91ef-c398fb6e069d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca5d9692a8c24d49925e1935ae2cd929, user_id=7fd31f1e4f3d440c9227eb69d9cd5ee3, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:17.996644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740e147-a34a-45d8-91ef-c398fb6e069d admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:18.012457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058f78-1ece-4e84-b3f9-110da1628ae3 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:18.013649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058f78-1ece-4e84-b3f9-110da1628ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:18.015414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf058f78-1ece-4e84-b3f9-110da1628ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:18.024623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 835/1639] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:17 2022] PUT /identity/v3/projects/ca5d9692a8c24d49925e1935ae2cd929/users/7fd31f1e4f3d440c9227eb69d9cd5ee3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:18.034789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964593d2-1b2e-438a-97e6-91d2302c9f0a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:18.035294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964593d2-1b2e-438a-97e6-91d2302c9f0a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:18.035849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964593d2-1b2e-438a-97e6-91d2302c9f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:18.060127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:18.069705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-bf058f78-1ece-4e84-b3f9-110da1628ae3 None None] MFA Rules not processed for user `b032ba8bfcc54f1d974ac32db61526ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:18.071903 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-bf058f78-1ece-4e84-b3f9-110da1628ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:18.106580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-964593d2-1b2e-438a-97e6-91d2302c9f0a None None] MFA Rules not processed for user `7fd31f1e4f3d440c9227eb69d9cd5ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:18.109593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-964593d2-1b2e-438a-97e6-91d2302c9f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:18.500282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 805/1640] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:18.500282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf058f78-1ece-4e84-b3f9-110da1628ae3 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:18.504617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 836/1641] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:18.504617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964593d2-1b2e-438a-97e6-91d2302c9f0a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:18.504617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964593d2-1b2e-438a-97e6-91d2302c9f0a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:18.504617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bccaa8-ef17-40d4-a076-dbb8b51f4666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:18.508706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bccaa8-ef17-40d4-a076-dbb8b51f4666 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:18.508706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bccaa8-ef17-40d4-a076-dbb8b51f4666 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:18.508706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bccaa8-ef17-40d4-a076-dbb8b51f4666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:18.525559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bccaa8-ef17-40d4-a076-dbb8b51f4666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:18.530064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bccaa8-ef17-40d4-a076-dbb8b51f4666 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:18.532185 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf058f78-1ece-4e84-b3f9-110da1628ae3 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:18.540895 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:18.542423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:18.543910 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:18.544842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] PATH_INFO: `/v3/users/95457e10373247639d89cc4fea72cac7` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:18.546422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] RBAC: Authorizing `identity:delete_user(user_id=95457e10373247639d89cc4fea72cac7)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:18.549758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:18.558151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 837/1642] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:18.700670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bccaa8-ef17-40d4-a076-dbb8b51f4666 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:18.721221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bccaa8-ef17-40d4-a076-dbb8b51f4666 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:18.726457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5ba77-25c0-482d-bf26-ad0d43f14e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:18.728198 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ba77-25c0-482d-bf26-ad0d43f14e54 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:18.728554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ba77-25c0-482d-bf26-ad0d43f14e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:18.728992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5ba77-25c0-482d-bf26-ad0d43f14e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:18.794325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5ba77-25c0-482d-bf26-ad0d43f14e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:18.798959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95457e10373247639d89cc4fea72cac7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:18.802374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5ba77-25c0-482d-bf26-ad0d43f14e54 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:18.831911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 838/1643] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:22:18.903199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95457e10373247639d89cc4fea72cac7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:18.904039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95457e10373247639d89cc4fea72cac7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:18.905186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95457e10373247639d89cc4fea72cac7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:18.920947 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95457e10373247639d89cc4fea72cac7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:18.925494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:18.957896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 806/1644] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:18 2022] DELETE /identity/v3/users/95457e10373247639d89cc4fea72cac7 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:22:18.962012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987071ca-114c-42d0-b138-3c55323424f3 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:18.962411 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987071ca-114c-42d0-b138-3c55323424f3 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:18.963346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987071ca-114c-42d0-b138-3c55323424f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:18.996985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-987071ca-114c-42d0-b138-3c55323424f3 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:18.998667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-987071ca-114c-42d0-b138-3c55323424f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:19.122479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 839/1645] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:19.132864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:19.134938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:19.213881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987071ca-114c-42d0-b138-3c55323424f3 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:19.220148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-987071ca-114c-42d0-b138-3c55323424f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:19.252463 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:19.257797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d4cff-133b-48d5-91c9-2f304284460e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:19.268431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b18ef-6bb5-49fc-b22f-50efbfaf356b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:19.269198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b18ef-6bb5-49fc-b22f-50efbfaf356b service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:19.269551 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b18ef-6bb5-49fc-b22f-50efbfaf356b service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:19.269897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b18ef-6bb5-49fc-b22f-50efbfaf356b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:19.288506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987071ca-114c-42d0-b138-3c55323424f3 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:19.292692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b18ef-6bb5-49fc-b22f-50efbfaf356b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:19.293275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8bd11-60bf-4344-a306-bf299df8ac2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:19.293904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8bd11-60bf-4344-a306-bf299df8ac2b service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:19.294359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8bd11-60bf-4344-a306-bf299df8ac2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:19.295884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b18ef-6bb5-49fc-b22f-50efbfaf356b service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:19.319926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8bd11-60bf-4344-a306-bf299df8ac2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:19.326169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fcb8bd11-60bf-4344-a306-bf299df8ac2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:19.359068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 807/1646] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:19.369482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b18ef-6bb5-49fc-b22f-50efbfaf356b service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:19.371529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d28b18ef-6bb5-49fc-b22f-50efbfaf356b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:19.400462 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b18ef-6bb5-49fc-b22f-50efbfaf356b service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:19.407201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89fe2e-b6f8-4203-8cbd-a72a9ad536c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:19.407201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89fe2e-b6f8-4203-8cbd-a72a9ad536c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:19.407201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89fe2e-b6f8-4203-8cbd-a72a9ad536c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:19.407201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89fe2e-b6f8-4203-8cbd-a72a9ad536c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:19.411311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-5b89fe2e-b6f8-4203-8cbd-a72a9ad536c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:19.545727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8bd11-60bf-4344-a306-bf299df8ac2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:19.547541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8bd11-60bf-4344-a306-bf299df8ac2b service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:19.562431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 840/1647] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:22:19.586878 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89fe2e-b6f8-4203-8cbd-a72a9ad536c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:19.588843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89fe2e-b6f8-4203-8cbd-a72a9ad536c3 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:19.614478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 808/1648] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:22:20.512972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8bd11-60bf-4344-a306-bf299df8ac2b service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:20.514886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fcb8bd11-60bf-4344-a306-bf299df8ac2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:20.541248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8bd11-60bf-4344-a306-bf299df8ac2b service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:20.549032 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:20.550120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:20.550635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:20.551015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] PATH_INFO: `/v3/projects/2d0163a702614d6a946843060e6c8f24` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:20.557114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d0163a702614d6a946843060e6c8f24)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:20.561046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:20.828106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d0163a702614d6a946843060e6c8f24', 'request_id': 'req-7d240303-22fb-4939-af69-c4a019544fb5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:20.866509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d0163a702614d6a946843060e6c8f24', 'request_id': 'req-7d240303-22fb-4939-af69-c4a019544fb5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:21.026263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d0163a702614d6a946843060e6c8f24', 'request_id': 'req-7d240303-22fb-4939-af69-c4a019544fb5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:21.040593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] The token cache is being invalidate because project 2d0163a702614d6a946843060e6c8f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:21.040885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:21.042057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7d240303-22fb-4939-af69-c4a019544fb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:21.044338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 841/1649] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:20 2022] DELETE /identity/v3/projects/2d0163a702614d6a946843060e6c8f24 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:21.049555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeb8c0-8f00-48f4-9e44-212659301fc4 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:21.050099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeb8c0-8f00-48f4-9e44-212659301fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:21.050099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfeb8c0-8f00-48f4-9e44-212659301fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:21.077912 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ccfeb8c0-8f00-48f4-9e44-212659301fc4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:21.078723 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ccfeb8c0-8f00-48f4-9e44-212659301fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:21.214394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 809/1650] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:21.223557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6812e5-f4bc-4348-94d2-e7acfe4c3372 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:21.224076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6812e5-f4bc-4348-94d2-e7acfe4c3372 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:21.224327 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6812e5-f4bc-4348-94d2-e7acfe4c3372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:21.276594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-eb6812e5-f4bc-4348-94d2-e7acfe4c3372 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:21.279191 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-eb6812e5-f4bc-4348-94d2-e7acfe4c3372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:21.305425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 842/1651] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:21.306052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeb8c0-8f00-48f4-9e44-212659301fc4 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:21.321160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfeb8c0-8f00-48f4-9e44-212659301fc4 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:21.332819 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bd55b-33e8-4769-a772-931f4e016429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:21.333444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bd55b-33e8-4769-a772-931f4e016429 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:21.333720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bd55b-33e8-4769-a772-931f4e016429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:21.334071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bd55b-33e8-4769-a772-931f4e016429 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:21.334666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bd55b-33e8-4769-a772-931f4e016429 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:21.334987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bd55b-33e8-4769-a772-931f4e016429 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:21.336587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bd55b-33e8-4769-a772-931f4e016429 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:21.389563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 810/1652] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:22:21.393978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6812e5-f4bc-4348-94d2-e7acfe4c3372 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:21.407132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6812e5-f4bc-4348-94d2-e7acfe4c3372 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:21.423948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fa9f8-54af-4571-a97d-869acfd988be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:21.424885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fa9f8-54af-4571-a97d-869acfd988be admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:21.425384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fa9f8-54af-4571-a97d-869acfd988be admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:21.425858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fa9f8-54af-4571-a97d-869acfd988be admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:21.427120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fa9f8-54af-4571-a97d-869acfd988be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:21.446509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fa9f8-54af-4571-a97d-869acfd988be admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:21.633348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 843/1653] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:21 2022] POST /identity/v3/projects => generated 405 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:21.643183 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bd55b-33e8-4769-a772-931f4e016429 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:21.682162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bd55b-33e8-4769-a772-931f4e016429 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:21.696872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:21.793328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0797e-ce7b-4f41-b914-1e40b3cc3d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:21.794265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0797e-ce7b-4f41-b914-1e40b3cc3d5e admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:21.795022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0797e-ce7b-4f41-b914-1e40b3cc3d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:21.795409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e0797e-ce7b-4f41-b914-1e40b3cc3d5e admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:21.796259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0797e-ce7b-4f41-b914-1e40b3cc3d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:21.798651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e0797e-ce7b-4f41-b914-1e40b3cc3d5e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:21.931624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 811/1654] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:21 2022] POST /identity/v3/users => generated 303 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:21.932931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fa9f8-54af-4571-a97d-869acfd988be admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:21.962630 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fa9f8-54af-4571-a97d-869acfd988be admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:21.992424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b14661-a0c0-4f32-ba31-2751c7f6a43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:21.993175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b14661-a0c0-4f32-ba31-2751c7f6a43c admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:22.009897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b14661-a0c0-4f32-ba31-2751c7f6a43c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:22.010331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b14661-a0c0-4f32-ba31-2751c7f6a43c admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:22.012834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b14661-a0c0-4f32-ba31-2751c7f6a43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:22.022958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b14661-a0c0-4f32-ba31-2751c7f6a43c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:22.082022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 844/1655] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:22.088890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0797e-ce7b-4f41-b914-1e40b3cc3d5e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:22.105505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e0797e-ce7b-4f41-b914-1e40b3cc3d5e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:22.117157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd6128-ca6d-4be6-a6d7-cf80edc3094e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:22.120254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd6128-ca6d-4be6-a6d7-cf80edc3094e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:22.121688 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd6128-ca6d-4be6-a6d7-cf80edc3094e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:22.123670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd6128-ca6d-4be6-a6d7-cf80edc3094e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:22.124994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd6128-ca6d-4be6-a6d7-cf80edc3094e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:22.127105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd6128-ca6d-4be6-a6d7-cf80edc3094e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:22.187061 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b14661-a0c0-4f32-ba31-2751c7f6a43c admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:22.198657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b14661-a0c0-4f32-ba31-2751c7f6a43c admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:22.209587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 812/1656] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:22.224283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dbae9-3e34-41bf-a9f0-22e93da80d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:22.225013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dbae9-3e34-41bf-a9f0-22e93da80d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:22.225294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dbae9-3e34-41bf-a9f0-22e93da80d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:22.225574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954dbae9-3e34-41bf-a9f0-22e93da80d38 admin admin] PATH_INFO: `/v3/projects/2a154551374b4160bc08095d20f0c49c/users/e6ac1d97e3c847af82a048fdac9d3560/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:22.278301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dbae9-3e34-41bf-a9f0-22e93da80d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a154551374b4160bc08095d20f0c49c, user_id=e6ac1d97e3c847af82a048fdac9d3560, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:22.281845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954dbae9-3e34-41bf-a9f0-22e93da80d38 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:22.285408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:22.372039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 845/1657] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:22 2022] PUT /identity/v3/projects/2a154551374b4160bc08095d20f0c49c/users/e6ac1d97e3c847af82a048fdac9d3560/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:22.379345 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df8b8a-bc63-4643-8bbb-63f4f8df1758 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:22.380242 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df8b8a-bc63-4643-8bbb-63f4f8df1758 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:22.380727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df8b8a-bc63-4643-8bbb-63f4f8df1758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:22.411942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-92df8b8a-bc63-4643-8bbb-63f4f8df1758 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:22.414287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-92df8b8a-bc63-4643-8bbb-63f4f8df1758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:22.581685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 813/1658] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:22.603158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dbae9-3e34-41bf-a9f0-22e93da80d38 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:22.607747 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-954dbae9-3e34-41bf-a9f0-22e93da80d38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:22.750807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954dbae9-3e34-41bf-a9f0-22e93da80d38 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:22.769225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1c37a-88eb-4f96-90e9-2258feafffb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:22.769875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1c37a-88eb-4f96-90e9-2258feafffb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:22.770164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1c37a-88eb-4f96-90e9-2258feafffb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:22.770429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1c37a-88eb-4f96-90e9-2258feafffb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:22.798582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1c37a-88eb-4f96-90e9-2258feafffb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:22.799997 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1c37a-88eb-4f96-90e9-2258feafffb0 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:22.826208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 846/1659] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:24.265778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df8b8a-bc63-4643-8bbb-63f4f8df1758 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:24.267552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-92df8b8a-bc63-4643-8bbb-63f4f8df1758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:24.300793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df8b8a-bc63-4643-8bbb-63f4f8df1758 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:24.320760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:24.320760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:24.320760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:24.320760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] PATH_INFO: `/v3/users/157b62754cf44d348387a142ec8a13a5` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:24.321998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=157b62754cf44d348387a142ec8a13a5)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:24.324418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:24.654604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '157b62754cf44d348387a142ec8a13a5'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:24.758888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '157b62754cf44d348387a142ec8a13a5'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:24.759660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '157b62754cf44d348387a142ec8a13a5'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:24.760399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '157b62754cf44d348387a142ec8a13a5'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:24.781838 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '157b62754cf44d348387a142ec8a13a5'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:24.840371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 814/1660] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:24 2022] DELETE /identity/v3/users/157b62754cf44d348387a142ec8a13a5 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:24.840664 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a1a11-8efd-44b9-9c25-bff34793d7f6 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:24.840664 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a1a11-8efd-44b9-9c25-bff34793d7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:24.840664 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a1a11-8efd-44b9-9c25-bff34793d7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:24.884963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-4a6a1a11-8efd-44b9-9c25-bff34793d7f6 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:24.891692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4a6a1a11-8efd-44b9-9c25-bff34793d7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:25.076366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 847/1661] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:25.089477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:25.091552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:25.107814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:25.237915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de623b-de87-41d9-a040-7de3153a3b2c admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:25.247567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6fa21-8aae-4f35-8d08-ab2f36a0e50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:25.248236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6fa21-8aae-4f35-8d08-ab2f36a0e50a service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:25.251448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6fa21-8aae-4f35-8d08-ab2f36a0e50a service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:25.251448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6fa21-8aae-4f35-8d08-ab2f36a0e50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:25.269532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6fa21-8aae-4f35-8d08-ab2f36a0e50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:25.271021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6fa21-8aae-4f35-8d08-ab2f36a0e50a service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:25.305270 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 815/1662] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:27.039357 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a1a11-8efd-44b9-9c25-bff34793d7f6 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:27.043901 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4a6a1a11-8efd-44b9-9c25-bff34793d7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:27.063303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6a1a11-8efd-44b9-9c25-bff34793d7f6 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:27.087191 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:27.088344 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:27.088797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:27.089176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] PATH_INFO: `/v3/projects/fce8cd8db6664823be80953567300846` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:27.093497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fce8cd8db6664823be80953567300846)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:27.098206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:27.308474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fce8cd8db6664823be80953567300846', 'request_id': 'req-52a33d9c-c68c-4921-8d96-da850afd876e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:27.315301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fce8cd8db6664823be80953567300846', 'request_id': 'req-52a33d9c-c68c-4921-8d96-da850afd876e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:27.452221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fce8cd8db6664823be80953567300846', 'request_id': 'req-52a33d9c-c68c-4921-8d96-da850afd876e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:27.469663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] The token cache is being invalidate because project fce8cd8db6664823be80953567300846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:27.469969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:27.475552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-52a33d9c-c68c-4921-8d96-da850afd876e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:27.490711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 848/1663] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:27 2022] DELETE /identity/v3/projects/fce8cd8db6664823be80953567300846 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:27.494584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f588b-a16d-4ee7-9c1c-d1ab93f16c1b None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:27.501098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f588b-a16d-4ee7-9c1c-d1ab93f16c1b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:27.501098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f588b-a16d-4ee7-9c1c-d1ab93f16c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:27.543631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-a11f588b-a16d-4ee7-9c1c-d1ab93f16c1b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:27.543631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a11f588b-a16d-4ee7-9c1c-d1ab93f16c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:27.874828 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 816/1664] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:27.880616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c170-dabf-408b-a1cd-8f8b5dbb7538 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:27.880616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c170-dabf-408b-a1cd-8f8b5dbb7538 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:27.880616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c170-dabf-408b-a1cd-8f8b5dbb7538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:27.947450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-fdd2c170-dabf-408b-a1cd-8f8b5dbb7538 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:27.949802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fdd2c170-dabf-408b-a1cd-8f8b5dbb7538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:27.979518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 849/1665] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:27.984147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f588b-a16d-4ee7-9c1c-d1ab93f16c1b None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:28.004028 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f588b-a16d-4ee7-9c1c-d1ab93f16c1b None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:28.038317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00719c2d-b409-494a-ad01-63c935799e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:28.038317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00719c2d-b409-494a-ad01-63c935799e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:28.038317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00719c2d-b409-494a-ad01-63c935799e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:28.038317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00719c2d-b409-494a-ad01-63c935799e41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:28.038619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00719c2d-b409-494a-ad01-63c935799e41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:28.040832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00719c2d-b409-494a-ad01-63c935799e41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:28.043945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00719c2d-b409-494a-ad01-63c935799e41 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:28.095885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 817/1666] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:28.100199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2c170-dabf-408b-a1cd-8f8b5dbb7538 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:28.153807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2c170-dabf-408b-a1cd-8f8b5dbb7538 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:28.188458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d9b1f-f9e4-4c8c-b34c-63fc5d201498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:28.190888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d9b1f-f9e4-4c8c-b34c-63fc5d201498 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:28.195069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d9b1f-f9e4-4c8c-b34c-63fc5d201498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:28.195069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d9b1f-f9e4-4c8c-b34c-63fc5d201498 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:28.201745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d9b1f-f9e4-4c8c-b34c-63fc5d201498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:28.206936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d9b1f-f9e4-4c8c-b34c-63fc5d201498 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:28.244779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf6bd1-902e-488f-aa61-c83a51095f5f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:28.244779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf6bd1-902e-488f-aa61-c83a51095f5f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:28.244779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf6bd1-902e-488f-aa61-c83a51095f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:28.278951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:28.294713 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-28cf6bd1-902e-488f-aa61-c83a51095f5f None None] MFA Rules not processed for user `22c867d61b384f9e8f692b053526f1b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:28.301566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-28cf6bd1-902e-488f-aa61-c83a51095f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:28.365187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 850/1667] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:28 2022] POST /identity/v3/projects => generated 373 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:28.379112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d9b1f-f9e4-4c8c-b34c-63fc5d201498 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:28.414849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 818/1668] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:28.425017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf6bd1-902e-488f-aa61-c83a51095f5f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:28.425017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-28cf6bd1-902e-488f-aa61-c83a51095f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:28.484933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:28.498538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d9b1f-f9e4-4c8c-b34c-63fc5d201498 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:28.513669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d5267-0fbc-454b-8155-99f439929424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:28.514313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5267-0fbc-454b-8155-99f439929424 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:28.514597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5267-0fbc-454b-8155-99f439929424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:28.514870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d5267-0fbc-454b-8155-99f439929424 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:28.515678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d5267-0fbc-454b-8155-99f439929424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:28.517857 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d5267-0fbc-454b-8155-99f439929424 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:28.568505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf6bd1-902e-488f-aa61-c83a51095f5f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:28.619677 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 851/1669] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:28 2022] POST /identity/v3/users => generated 287 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:28.626419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d5267-0fbc-454b-8155-99f439929424 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:28.650286 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d5267-0fbc-454b-8155-99f439929424 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:28.661450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fd8f7-6648-4d68-a344-c2828ee6bbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:28.662643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fd8f7-6648-4d68-a344-c2828ee6bbfe admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:28.663248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fd8f7-6648-4d68-a344-c2828ee6bbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:28.663674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6fd8f7-6648-4d68-a344-c2828ee6bbfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:28.664503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fd8f7-6648-4d68-a344-c2828ee6bbfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:28.670045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6fd8f7-6648-4d68-a344-c2828ee6bbfe admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:28.695811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 852/1670] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:28.704079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fd8f7-6648-4d68-a344-c2828ee6bbfe admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:28.707488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140657b-f994-4460-a0a0-af2c06c9b831 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:28.711845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140657b-f994-4460-a0a0-af2c06c9b831 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:28.712213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140657b-f994-4460-a0a0-af2c06c9b831 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:28.714095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140657b-f994-4460-a0a0-af2c06c9b831 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:28.723099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6fd8f7-6648-4d68-a344-c2828ee6bbfe admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:28.734432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3c082-fc91-4949-96c5-b188adbee282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:28.734432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c082-fc91-4949-96c5-b188adbee282 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:28.734432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c082-fc91-4949-96c5-b188adbee282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:28.734792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3c082-fc91-4949-96c5-b188adbee282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:28.735656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3c082-fc91-4949-96c5-b188adbee282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:28.747290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3c082-fc91-4949-96c5-b188adbee282 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:28.789221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 853/1671] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:28.790627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293cb69-d483-4761-9295-e965b305811d None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:28.790967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293cb69-d483-4761-9295-e965b305811d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:28.791246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d293cb69-d483-4761-9295-e965b305811d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:28.873495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-d293cb69-d483-4761-9295-e965b305811d None None] MFA Rules not processed for user `e6ac1d97e3c847af82a048fdac9d3560`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:28.874739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140657b-f994-4460-a0a0-af2c06c9b831 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:28.875271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d293cb69-d483-4761-9295-e965b305811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:28.876404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140657b-f994-4460-a0a0-af2c06c9b831 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:28.904428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 819/1672] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:22:28.904428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80884ec3-376d-415d-90a3-a0dfd754aef5 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:28.904428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80884ec3-376d-415d-90a3-a0dfd754aef5 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:28.904428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80884ec3-376d-415d-90a3-a0dfd754aef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:29.036481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 854/1673] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:28 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:29.037642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293cb69-d483-4761-9295-e965b305811d None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:29.053353 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d293cb69-d483-4761-9295-e965b305811d None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:29.060783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f203d-1866-4a0b-ad10-a2ea14a198db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:29.061768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f203d-1866-4a0b-ad10-a2ea14a198db admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:29.062356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f203d-1866-4a0b-ad10-a2ea14a198db admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:29.062781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f203d-1866-4a0b-ad10-a2ea14a198db admin admin] PATH_INFO: `/v3/projects/7aa97db74be7441a97945ce265965e8f/users/dc721ee2b6864ffb9413db3612372a1e/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:29.075642 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-80884ec3-376d-415d-90a3-a0dfd754aef5 None None] MFA Rules not processed for user `8430cbddaa1448f6ade09a082fa29b13`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:29.077238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-80884ec3-376d-415d-90a3-a0dfd754aef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:29.134944 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f203d-1866-4a0b-ad10-a2ea14a198db admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa97db74be7441a97945ce265965e8f, user_id=dc721ee2b6864ffb9413db3612372a1e, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:29.152245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f203d-1866-4a0b-ad10-a2ea14a198db admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:29.250558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:29.276622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 855/1674] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:29 2022] PUT /identity/v3/projects/7aa97db74be7441a97945ce265965e8f/users/dc721ee2b6864ffb9413db3612372a1e/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:29.277647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f203d-1866-4a0b-ad10-a2ea14a198db admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:29.281900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b09f203d-1866-4a0b-ad10-a2ea14a198db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:29.321573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 820/1675] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:29.330203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93812361-c521-45a1-bfc1-8c4d5fc57c2c None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:29.330505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93812361-c521-45a1-bfc1-8c4d5fc57c2c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:29.330764 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93812361-c521-45a1-bfc1-8c4d5fc57c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:29.392421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-93812361-c521-45a1-bfc1-8c4d5fc57c2c None None] MFA Rules not processed for user `dc721ee2b6864ffb9413db3612372a1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:29.393491 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-93812361-c521-45a1-bfc1-8c4d5fc57c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:29.496260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f203d-1866-4a0b-ad10-a2ea14a198db admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:29.517070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7e4d-1db8-4367-81f7-84f976b6ca37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:29.519743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7e4d-1db8-4367-81f7-84f976b6ca37 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:29.521875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7e4d-1db8-4367-81f7-84f976b6ca37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:29.521875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d7e4d-1db8-4367-81f7-84f976b6ca37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:29.522741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d7e4d-1db8-4367-81f7-84f976b6ca37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:29.523649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 821/1676] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:29.529025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93812361-c521-45a1-bfc1-8c4d5fc57c2c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:29.530509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d7e4d-1db8-4367-81f7-84f976b6ca37 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:29.531643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-93812361-c521-45a1-bfc1-8c4d5fc57c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:29.650492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93812361-c521-45a1-bfc1-8c4d5fc57c2c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:29.658428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332e1b5-1c92-49a7-aaa2-46b5732d44c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:29.659647 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e1b5-1c92-49a7-aaa2-46b5732d44c5 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:29.660111 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e1b5-1c92-49a7-aaa2-46b5732d44c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:29.660543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7332e1b5-1c92-49a7-aaa2-46b5732d44c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:29.679541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 856/1677] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:29 2022] POST /identity/v3/projects => generated 405 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:29.680966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7e4d-1db8-4367-81f7-84f976b6ca37 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:29.795832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e1b5-1c92-49a7-aaa2-46b5732d44c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:29.799913 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7332e1b5-1c92-49a7-aaa2-46b5732d44c5 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:29.804884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d7e4d-1db8-4367-81f7-84f976b6ca37 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:29.812438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1a83b-0ea1-4f73-b0af-897c50ba722a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:29.821169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1a83b-0ea1-4f73-b0af-897c50ba722a admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:29.821542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1a83b-0ea1-4f73-b0af-897c50ba722a admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:29.821826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1a83b-0ea1-4f73-b0af-897c50ba722a admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:29.822570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1a83b-0ea1-4f73-b0af-897c50ba722a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:29.822821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 822/1678] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:29.823758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332e1b5-1c92-49a7-aaa2-46b5732d44c5 service nova] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:29.824951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1a83b-0ea1-4f73-b0af-897c50ba722a admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:29.842477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7332e1b5-1c92-49a7-aaa2-46b5732d44c5 service nova] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:29.847944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b4ad1-0bd3-452b-a7ea-040dcf8e67ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:29.849215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b4ad1-0bd3-452b-a7ea-040dcf8e67ff admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:29.850269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b4ad1-0bd3-452b-a7ea-040dcf8e67ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:29.850553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6b4ad1-0bd3-452b-a7ea-040dcf8e67ff admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:29.851602 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b4ad1-0bd3-452b-a7ea-040dcf8e67ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:29.857376 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6b4ad1-0bd3-452b-a7ea-040dcf8e67ff admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:29.875239 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:29.944080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 857/1679] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:29 2022] POST /identity/v3/projects => generated 373 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:29.955668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1a83b-0ea1-4f73-b0af-897c50ba722a admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:29.993296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 823/1680] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:29 2022] POST /identity/v3/users => generated 309 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:29.998909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b4ad1-0bd3-452b-a7ea-040dcf8e67ff admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:30.024525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6b4ad1-0bd3-452b-a7ea-040dcf8e67ff admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:30.088924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1a83b-0ea1-4f73-b0af-897c50ba722a admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:30.103593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0051da-ea56-49d7-bf55-fa9976be15f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:30.104265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0051da-ea56-49d7-bf55-fa9976be15f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.104552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0051da-ea56-49d7-bf55-fa9976be15f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.104829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0051da-ea56-49d7-bf55-fa9976be15f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.106039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0051da-ea56-49d7-bf55-fa9976be15f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:30.109404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0051da-ea56-49d7-bf55-fa9976be15f6 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:30.155571 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716252d-c67b-4b50-9af4-37c25b402f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:30.159488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716252d-c67b-4b50-9af4-37c25b402f40 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.159488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716252d-c67b-4b50-9af4-37c25b402f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.159488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716252d-c67b-4b50-9af4-37c25b402f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.159845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716252d-c67b-4b50-9af4-37c25b402f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:30.161620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716252d-c67b-4b50-9af4-37c25b402f40 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:30.205704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 824/1681] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:30.215614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716252d-c67b-4b50-9af4-37c25b402f40 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:30.221820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 858/1682] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:29 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:30.223618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0051da-ea56-49d7-bf55-fa9976be15f6 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:30.247195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0051da-ea56-49d7-bf55-fa9976be15f6 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:30.247693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716252d-c67b-4b50-9af4-37c25b402f40 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:30.265218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84274b10-4f73-4f9d-b813-54a16b2e3e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:30.266227 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d99f1e-a726-4802-8bfd-44f47cbe2d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:30.267043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d99f1e-a726-4802-8bfd-44f47cbe2d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.267194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274b10-4f73-4f9d-b813-54a16b2e3e32 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.267333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d99f1e-a726-4802-8bfd-44f47cbe2d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.267479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274b10-4f73-4f9d-b813-54a16b2e3e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.267757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84274b10-4f73-4f9d-b813-54a16b2e3e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.268394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d99f1e-a726-4802-8bfd-44f47cbe2d36 admin admin] PATH_INFO: `/v3/projects/31cf563be55b4698ab62606fb8150302/users/27741abb99384bb7944d4034e211c916/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.270124 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84274b10-4f73-4f9d-b813-54a16b2e3e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:30.278612 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84274b10-4f73-4f9d-b813-54a16b2e3e32 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:30.282855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:30.309633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 859/1683] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:30.313588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84274b10-4f73-4f9d-b813-54a16b2e3e32 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:30.335749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d99f1e-a726-4802-8bfd-44f47cbe2d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31cf563be55b4698ab62606fb8150302, user_id=27741abb99384bb7944d4034e211c916, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:30.336736 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84274b10-4f73-4f9d-b813-54a16b2e3e32 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:30.339420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d99f1e-a726-4802-8bfd-44f47cbe2d36 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:30.349555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257844c-13c9-486a-b48a-20cb46468986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:30.350252 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257844c-13c9-486a-b48a-20cb46468986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.350536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257844c-13c9-486a-b48a-20cb46468986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.350814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257844c-13c9-486a-b48a-20cb46468986 admin admin] PATH_INFO: `/v3/projects/c29cad19c58a4c52a9f94b082c252d66/users/cbf23fceb2a04255ac9d1cce6da572b8/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.386331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257844c-13c9-486a-b48a-20cb46468986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29cad19c58a4c52a9f94b082c252d66, user_id=cbf23fceb2a04255ac9d1cce6da572b8, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:30.387789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257844c-13c9-486a-b48a-20cb46468986 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:30.462319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 825/1684] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:30 2022] PUT /identity/v3/projects/31cf563be55b4698ab62606fb8150302/users/27741abb99384bb7944d4034e211c916/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:30.463272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d99f1e-a726-4802-8bfd-44f47cbe2d36 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:30.498528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 860/1685] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:30 2022] PUT /identity/v3/projects/c29cad19c58a4c52a9f94b082c252d66/users/cbf23fceb2a04255ac9d1cce6da572b8/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:30.504008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257844c-13c9-486a-b48a-20cb46468986 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:30.586630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d99f1e-a726-4802-8bfd-44f47cbe2d36 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:30.616760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257844c-13c9-486a-b48a-20cb46468986 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:30.628723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691827b-f244-47f5-88fc-429d784a3902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:30.633214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691827b-f244-47f5-88fc-429d784a3902 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.633214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691827b-f244-47f5-88fc-429d784a3902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.633214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691827b-f244-47f5-88fc-429d784a3902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.633214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691827b-f244-47f5-88fc-429d784a3902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:30.633214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d691827b-f244-47f5-88fc-429d784a3902 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:30.644555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 861/1686] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:22:30.646173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b898aaa-b760-4ff4-9b40-4ef640f134ec None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.646900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b898aaa-b760-4ff4-9b40-4ef640f134ec None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.647132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b898aaa-b760-4ff4-9b40-4ef640f134ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.656718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ebb5e-5c14-44bd-8c6b-3467feaf44ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:30.657359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ebb5e-5c14-44bd-8c6b-3467feaf44ab admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.657692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ebb5e-5c14-44bd-8c6b-3467feaf44ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.661731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9ebb5e-5c14-44bd-8c6b-3467feaf44ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.661731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ebb5e-5c14-44bd-8c6b-3467feaf44ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:30.661731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9ebb5e-5c14-44bd-8c6b-3467feaf44ab admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:30.685848 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 826/1687] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:30.687264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ebb5e-5c14-44bd-8c6b-3467feaf44ab admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:30.696451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8b898aaa-b760-4ff4-9b40-4ef640f134ec None None] MFA Rules not processed for user `2eff82d212164719aacb10143d365259`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:30.697984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8b898aaa-b760-4ff4-9b40-4ef640f134ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:30.712492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9ebb5e-5c14-44bd-8c6b-3467feaf44ab admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:30.720663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb97c4b-4c55-47ce-b289-9f0ab6747954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:30.721459 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb97c4b-4c55-47ce-b289-9f0ab6747954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.722179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb97c4b-4c55-47ce-b289-9f0ab6747954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.722462 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb97c4b-4c55-47ce-b289-9f0ab6747954 admin admin] PATH_INFO: `/v3/projects/c29cad19c58a4c52a9f94b082c252d66/users/cbf23fceb2a04255ac9d1cce6da572b8/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.725335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb97c4b-4c55-47ce-b289-9f0ab6747954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29cad19c58a4c52a9f94b082c252d66, user_id=cbf23fceb2a04255ac9d1cce6da572b8, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:30.728427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb97c4b-4c55-47ce-b289-9f0ab6747954 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:30.743851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 827/1688] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:30 2022] PUT /identity/v3/projects/c29cad19c58a4c52a9f94b082c252d66/users/cbf23fceb2a04255ac9d1cce6da572b8/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:30.744818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb97c4b-4c55-47ce-b289-9f0ab6747954 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:30.807921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 862/1689] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:30.810133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89ff48-504d-4d1a-931b-382da4fad782 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.810557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89ff48-504d-4d1a-931b-382da4fad782 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.810896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c89ff48-504d-4d1a-931b-382da4fad782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.842527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-4c89ff48-504d-4d1a-931b-382da4fad782 None None] MFA Rules not processed for user `cbf23fceb2a04255ac9d1cce6da572b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:30.844285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4c89ff48-504d-4d1a-931b-382da4fad782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:30.870918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb97c4b-4c55-47ce-b289-9f0ab6747954 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:30.875849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3aebd-a946-40ee-b0cb-58f6d42671de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:30.876607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3aebd-a946-40ee-b0cb-58f6d42671de admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.876953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3aebd-a946-40ee-b0cb-58f6d42671de admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.877303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3aebd-a946-40ee-b0cb-58f6d42671de admin admin] PATH_INFO: `/v3/projects/31cf563be55b4698ab62606fb8150302/users/27741abb99384bb7944d4034e211c916/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.879433 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3aebd-a946-40ee-b0cb-58f6d42671de admin admin] RBAC: Authorizing `identity:create_grant(project_id=31cf563be55b4698ab62606fb8150302, user_id=27741abb99384bb7944d4034e211c916, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:30.893429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3aebd-a946-40ee-b0cb-58f6d42671de admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:30.913945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 828/1690] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:30 2022] PUT /identity/v3/projects/31cf563be55b4698ab62606fb8150302/users/27741abb99384bb7944d4034e211c916/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:30.915404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb98a04-f880-4627-ba97-ab0e13554edc None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:30.915695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb98a04-f880-4627-ba97-ab0e13554edc None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:30.916000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb98a04-f880-4627-ba97-ab0e13554edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:30.951544 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 863/1691] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:30.965746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89ff48-504d-4d1a-931b-382da4fad782 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:30.967397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4c89ff48-504d-4d1a-931b-382da4fad782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:30.968888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-9bb98a04-f880-4627-ba97-ab0e13554edc None None] MFA Rules not processed for user `2eff82d212164719aacb10143d365259`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:30.970280 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9bb98a04-f880-4627-ba97-ab0e13554edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:31.022447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:31.060491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c89ff48-504d-4d1a-931b-382da4fad782 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:31.076346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 829/1692] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:31.097077 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb98a04-f880-4627-ba97-ab0e13554edc None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:31.134048 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0ee8f-0b82-4dd9-95b6-71650c17d2fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:31.137347 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0ee8f-0b82-4dd9-95b6-71650c17d2fc service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:31.138040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0ee8f-0b82-4dd9-95b6-71650c17d2fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:31.138620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0ee8f-0b82-4dd9-95b6-71650c17d2fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:31.204745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb98a04-f880-4627-ba97-ab0e13554edc None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:31.212559 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6c5d4-3ebb-41c9-90b3-eaf2ed4a6bf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:31.215261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c5d4-3ebb-41c9-90b3-eaf2ed4a6bf7 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:31.215261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c5d4-3ebb-41c9-90b3-eaf2ed4a6bf7 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:31.215261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6c5d4-3ebb-41c9-90b3-eaf2ed4a6bf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:31.226882 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6c5d4-3ebb-41c9-90b3-eaf2ed4a6bf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:31.228544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6c5d4-3ebb-41c9-90b3-eaf2ed4a6bf7 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:31.242655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0ee8f-0b82-4dd9-95b6-71650c17d2fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:31.243363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 830/1693] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:31.250580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0ee8f-0b82-4dd9-95b6-71650c17d2fc service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:31.276975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 864/1694] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:31.634573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:32.104775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52311230-177e-4610-90b5-37cd8ddcdd92 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:32.105312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52311230-177e-4610-90b5-37cd8ddcdd92 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:32.108025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52311230-177e-4610-90b5-37cd8ddcdd92 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:32.111124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 831/1695] 10.222.0.15 () {54 vars in 912 bytes} [Wed Sep 7 16:22:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:22:32.116224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b0d72-5ce9-47f4-973e-a71e93cce1cb None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:32.116701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b0d72-5ce9-47f4-973e-a71e93cce1cb None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:32.117120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b0d72-5ce9-47f4-973e-a71e93cce1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:32.170802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-a28b0d72-5ce9-47f4-973e-a71e93cce1cb None None] MFA Rules not processed for user `b6a6b52462624e7f84f479144353b51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:32.170802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a28b0d72-5ce9-47f4-973e-a71e93cce1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:32.266529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 865/1696] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:22:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:32.289443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52311230-177e-4610-90b5-37cd8ddcdd92 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:32.291783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-52311230-177e-4610-90b5-37cd8ddcdd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:32.333102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52311230-177e-4610-90b5-37cd8ddcdd92 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:32.337873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad7c50b-9c81-4d4c-bc46-d96f9fbb455e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:32.338494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7c50b-9c81-4d4c-bc46-d96f9fbb455e service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:32.340544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7c50b-9c81-4d4c-bc46-d96f9fbb455e service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:32.340544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad7c50b-9c81-4d4c-bc46-d96f9fbb455e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:32.348367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7c50b-9c81-4d4c-bc46-d96f9fbb455e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:32.349835 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad7c50b-9c81-4d4c-bc46-d96f9fbb455e service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:32.361561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 832/1697] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:32.688243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b0d72-5ce9-47f4-973e-a71e93cce1cb None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:32.849389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b0d72-5ce9-47f4-973e-a71e93cce1cb None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:32.856989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:32.863058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:32.863058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:32.863058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] PATH_INFO: `/v3/users/dc721ee2b6864ffb9413db3612372a1e` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:32.863058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc721ee2b6864ffb9413db3612372a1e)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:32.863850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: WARNING py.warnings [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:22:32.863850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: warnings.warn(msg) Sep 07 16:22:32.863850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: Sep 07 16:22:32.866352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:33.187398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc721ee2b6864ffb9413db3612372a1e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:33.352509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc721ee2b6864ffb9413db3612372a1e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:33.360925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc721ee2b6864ffb9413db3612372a1e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:33.360925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc721ee2b6864ffb9413db3612372a1e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:33.403737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc721ee2b6864ffb9413db3612372a1e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:33.453882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 866/1698] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:32 2022] DELETE /identity/v3/users/dc721ee2b6864ffb9413db3612372a1e => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:33.457587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863f21f-0cea-4237-8ec4-41b15e9113a7 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:33.458136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863f21f-0cea-4237-8ec4-41b15e9113a7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:33.458136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863f21f-0cea-4237-8ec4-41b15e9113a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:33.476623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:33.477968 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:33.550164 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0863f21f-0cea-4237-8ec4-41b15e9113a7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:33.556548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0863f21f-0cea-4237-8ec4-41b15e9113a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:33.572820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:33.591368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:33.737318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 833/1699] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:33.738680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91e093-05c4-4241-9ee0-41896097180d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:33.738987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91e093-05c4-4241-9ee0-41896097180d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:33.739251 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91e093-05c4-4241-9ee0-41896097180d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:33.750571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e6e3b-aa7c-48f6-9d81-005ea64b53f9 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:33.759062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7b923-5c85-489b-9388-9c70a0298029 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:33.759062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7b923-5c85-489b-9388-9c70a0298029 service placement] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:33.759244 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7b923-5c85-489b-9388-9c70a0298029 service placement] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:33.759491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7b923-5c85-489b-9388-9c70a0298029 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:33.763022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d0a7b923-5c85-489b-9388-9c70a0298029 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:33.790404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7b923-5c85-489b-9388-9c70a0298029 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:33.795703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7b923-5c85-489b-9388-9c70a0298029 service placement] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:33.803480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ed91e093-05c4-4241-9ee0-41896097180d None None] MFA Rules not processed for user `27741abb99384bb7944d4034e211c916`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:33.808234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ed91e093-05c4-4241-9ee0-41896097180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:33.845655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 867/1700] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:22:33.852297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7b923-5c85-489b-9388-9c70a0298029 service placement] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:33.860613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d0a7b923-5c85-489b-9388-9c70a0298029 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:34.025308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 834/1701] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:34.026842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a123c-325c-43ef-b60b-f4626ea79bf7 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:34.029751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a123c-325c-43ef-b60b-f4626ea79bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:34.029751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a123c-325c-43ef-b60b-f4626ea79bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:34.045368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7b923-5c85-489b-9388-9c70a0298029 service placement] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:34.056724 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820dccf-77ae-405e-a6ab-c4028b10ee0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:34.059158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820dccf-77ae-405e-a6ab-c4028b10ee0a service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:34.060383 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820dccf-77ae-405e-a6ab-c4028b10ee0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:34.061112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820dccf-77ae-405e-a6ab-c4028b10ee0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:34.070983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-327a123c-325c-43ef-b60b-f4626ea79bf7 None None] MFA Rules not processed for user `22c867d61b384f9e8f692b053526f1b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:34.072973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-327a123c-325c-43ef-b60b-f4626ea79bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:34.088166 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820dccf-77ae-405e-a6ab-c4028b10ee0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:34.090658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820dccf-77ae-405e-a6ab-c4028b10ee0a service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:34.135504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 868/1702] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:34.213326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 835/1703] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:35.013388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86e42f-2ddb-493c-a2bf-5a620f62c428 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:35.014010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86e42f-2ddb-493c-a2bf-5a620f62c428 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:35.014391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86e42f-2ddb-493c-a2bf-5a620f62c428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:35.038899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8f86e42f-2ddb-493c-a2bf-5a620f62c428 None None] MFA Rules not processed for user `8430cbddaa1448f6ade09a082fa29b13`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:35.040261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8f86e42f-2ddb-493c-a2bf-5a620f62c428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:35.124750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 869/1704] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:35.291332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a123c-325c-43ef-b60b-f4626ea79bf7 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:35.292872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-327a123c-325c-43ef-b60b-f4626ea79bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:35.315106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a123c-325c-43ef-b60b-f4626ea79bf7 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:35.323005 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:35.333532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:35.334118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:35.334404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] PATH_INFO: `/v3/projects/7aa97db74be7441a97945ce265965e8f` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:35.335933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa97db74be7441a97945ce265965e8f)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:35.336960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: WARNING py.warnings [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 16:22:35.336960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: warnings.warn(msg) Sep 07 16:22:35.336960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: Sep 07 16:22:35.338161 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:35.468009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:35.492075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa97db74be7441a97945ce265965e8f', 'request_id': 'req-b88363ee-ecdf-456a-9546-3d951cda7d01'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:35.538671 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:35.572704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa97db74be7441a97945ce265965e8f', 'request_id': 'req-b88363ee-ecdf-456a-9546-3d951cda7d01'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:35.609965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa97db74be7441a97945ce265965e8f', 'request_id': 'req-b88363ee-ecdf-456a-9546-3d951cda7d01'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:35.645427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] The token cache is being invalidate because project 7aa97db74be7441a97945ce265965e8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:35.649081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:35.651802 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b88363ee-ecdf-456a-9546-3d951cda7d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:35.654192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 836/1705] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:35 2022] DELETE /identity/v3/projects/7aa97db74be7441a97945ce265965e8f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:35.658459 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86e42f-2ddb-493c-a2bf-5a620f62c428 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:35.679657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86e42f-2ddb-493c-a2bf-5a620f62c428 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:35.761127 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e1457-1327-4dd6-9458-6ede41993bcd None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:35.761437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e1457-1327-4dd6-9458-6ede41993bcd None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:35.761694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e1457-1327-4dd6-9458-6ede41993bcd None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:35.763574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 837/1706] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:22:35.777932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b75cbe-5225-43e6-b088-c5f55167931f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:35.778419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b75cbe-5225-43e6-b088-c5f55167931f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:35.778833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b75cbe-5225-43e6-b088-c5f55167931f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:35.825020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-d0b75cbe-5225-43e6-b088-c5f55167931f None None] MFA Rules not processed for user `5fdfb0bf09db406ab5c09fbef887cbab`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:35.828417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d0b75cbe-5225-43e6-b088-c5f55167931f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:35.837726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:35.839700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:35.840273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:35.841105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] PATH_INFO: `/v3/users/cbf23fceb2a04255ac9d1cce6da572b8` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:35.844885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbf23fceb2a04255ac9d1cce6da572b8)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:35.846989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:36.038775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 838/1707] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:22:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:36.038775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90db524-4052-41e3-8ded-cb2363bb2581 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:36.038775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90db524-4052-41e3-8ded-cb2363bb2581 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:36.038775 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90db524-4052-41e3-8ded-cb2363bb2581 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:36.056062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 839/1708] 10.222.0.15 () {54 vars in 912 bytes} [Wed Sep 7 16:22:36 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:22:36.061448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90db524-4052-41e3-8ded-cb2363bb2581 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:36.062058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c90db524-4052-41e3-8ded-cb2363bb2581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:36.090672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90db524-4052-41e3-8ded-cb2363bb2581 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:36.102204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caa7872-fe7a-43ba-af22-3c89d9091746 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:36.107324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa7872-fe7a-43ba-af22-3c89d9091746 service placement] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:36.107324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa7872-fe7a-43ba-af22-3c89d9091746 service placement] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:36.107324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caa7872-fe7a-43ba-af22-3c89d9091746 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:36.126071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa7872-fe7a-43ba-af22-3c89d9091746 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:36.128150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caa7872-fe7a-43ba-af22-3c89d9091746 service placement] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:36.164535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 840/1709] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:36.165158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbf23fceb2a04255ac9d1cce6da572b8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:36.170268 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ad99f-03ca-4a3f-8d81-3a0ec5a07f1e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:36.171061 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ad99f-03ca-4a3f-8d81-3a0ec5a07f1e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:36.171524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ad99f-03ca-4a3f-8d81-3a0ec5a07f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:36.212702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:36.220283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-4f9ad99f-03ca-4a3f-8d81-3a0ec5a07f1e None None] MFA Rules not processed for user `b6a6b52462624e7f84f479144353b51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:36.222257 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4f9ad99f-03ca-4a3f-8d81-3a0ec5a07f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:36.251910 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbf23fceb2a04255ac9d1cce6da572b8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:36.253639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbf23fceb2a04255ac9d1cce6da572b8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:36.254481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbf23fceb2a04255ac9d1cce6da572b8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:36.267003 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbf23fceb2a04255ac9d1cce6da572b8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:36.321415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 870/1710] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:35 2022] DELETE /identity/v3/users/cbf23fceb2a04255ac9d1cce6da572b8 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:22:36.325916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 841/1711] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:22:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:36.357079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:36.359146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:36.444439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb9445-8a82-4252-8d92-55725bf27ab0 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:36.448490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debf8f2-19c5-43f7-8fa4-e714607bbff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:36.449114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf8f2-19c5-43f7-8fa4-e714607bbff9 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:36.449390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf8f2-19c5-43f7-8fa4-e714607bbff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:36.449667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debf8f2-19c5-43f7-8fa4-e714607bbff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:36.457683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debf8f2-19c5-43f7-8fa4-e714607bbff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:36.459161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debf8f2-19c5-43f7-8fa4-e714607bbff9 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:36.471980 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 871/1712] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:37.203208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ad99f-03ca-4a3f-8d81-3a0ec5a07f1e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:37.204831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4f9ad99f-03ca-4a3f-8d81-3a0ec5a07f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:37.254580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ad99f-03ca-4a3f-8d81-3a0ec5a07f1e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:37.331015 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea69b0-d9c7-42c9-b7ac-b623360bc2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:37.332635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:37.334399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea69b0-d9c7-42c9-b7ac-b623360bc2aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:37.335024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea69b0-d9c7-42c9-b7ac-b623360bc2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:37.335482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea69b0-d9c7-42c9-b7ac-b623360bc2aa admin admin] PATH_INFO: `/v3/users/e6ac1d97e3c847af82a048fdac9d3560` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:37.337330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea69b0-d9c7-42c9-b7ac-b623360bc2aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ac1d97e3c847af82a048fdac9d3560)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:37.339629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea69b0-d9c7-42c9-b7ac-b623360bc2aa admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:37.479243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debf8f2-19c5-43f7-8fa4-e714607bbff9 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:37.480731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6debf8f2-19c5-43f7-8fa4-e714607bbff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:37.527734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debf8f2-19c5-43f7-8fa4-e714607bbff9 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:37.547290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:37.548672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:37.550023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:37.550023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] PATH_INFO: `/v3/projects/c29cad19c58a4c52a9f94b082c252d66` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:37.553445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29cad19c58a4c52a9f94b082c252d66)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:37.558107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:37.648035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-55ea69b0-d9c7-42c9-b7ac-b623360bc2aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ac1d97e3c847af82a048fdac9d3560'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:37.781337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29cad19c58a4c52a9f94b082c252d66', 'request_id': 'req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:37.793825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-55ea69b0-d9c7-42c9-b7ac-b623360bc2aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ac1d97e3c847af82a048fdac9d3560'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:37.793825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-55ea69b0-d9c7-42c9-b7ac-b623360bc2aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ac1d97e3c847af82a048fdac9d3560'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:37.793825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-55ea69b0-d9c7-42c9-b7ac-b623360bc2aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ac1d97e3c847af82a048fdac9d3560'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:37.804841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-55ea69b0-d9c7-42c9-b7ac-b623360bc2aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ac1d97e3c847af82a048fdac9d3560'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:37.806522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29cad19c58a4c52a9f94b082c252d66', 'request_id': 'req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:37.878192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 842/1713] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:37 2022] DELETE /identity/v3/users/e6ac1d97e3c847af82a048fdac9d3560 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:37.979056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29cad19c58a4c52a9f94b082c252d66', 'request_id': 'req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:38.008749 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] The token cache is being invalidate because project c29cad19c58a4c52a9f94b082c252d66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:38.009272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:38.018432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-23bc8833-bf82-4fa9-83f6-41fc1cf9b9be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:38.029573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 872/1714] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:37 2022] DELETE /identity/v3/projects/c29cad19c58a4c52a9f94b082c252d66 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:38.040930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33318dc1-4b8d-4ea4-a10f-6dd1cf0f9336 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:38.041438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33318dc1-4b8d-4ea4-a10f-6dd1cf0f9336 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:38.041984 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33318dc1-4b8d-4ea4-a10f-6dd1cf0f9336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:38.081720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-33318dc1-4b8d-4ea4-a10f-6dd1cf0f9336 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:38.085264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-33318dc1-4b8d-4ea4-a10f-6dd1cf0f9336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:38.180285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 843/1715] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:38.188222 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded69d2-c959-42f3-96e4-84305abdb0ab None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:38.188805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded69d2-c959-42f3-96e4-84305abdb0ab None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:38.189237 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded69d2-c959-42f3-96e4-84305abdb0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:38.212037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-fded69d2-c959-42f3-96e4-84305abdb0ab None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:38.213324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fded69d2-c959-42f3-96e4-84305abdb0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:38.224722 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 873/1716] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:38.229079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33318dc1-4b8d-4ea4-a10f-6dd1cf0f9336 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:38.239517 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33318dc1-4b8d-4ea4-a10f-6dd1cf0f9336 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:38.257465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47569aa-1a31-4a81-9eab-4cd202ea8ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:38.258104 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47569aa-1a31-4a81-9eab-4cd202ea8ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:38.258387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47569aa-1a31-4a81-9eab-4cd202ea8ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:38.258661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47569aa-1a31-4a81-9eab-4cd202ea8ca0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:38.259322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47569aa-1a31-4a81-9eab-4cd202ea8ca0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:38.259628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47569aa-1a31-4a81-9eab-4cd202ea8ca0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:38.261457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47569aa-1a31-4a81-9eab-4cd202ea8ca0 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:38.301206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 844/1717] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:38.306133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded69d2-c959-42f3-96e4-84305abdb0ab None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:38.318611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded69d2-c959-42f3-96e4-84305abdb0ab None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:38.336832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e40ae-5836-4c0a-b6af-439d663afd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:38.337474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e40ae-5836-4c0a-b6af-439d663afd91 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:38.337843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e40ae-5836-4c0a-b6af-439d663afd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:38.338292 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56e40ae-5836-4c0a-b6af-439d663afd91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:38.345020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e40ae-5836-4c0a-b6af-439d663afd91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:38.355476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56e40ae-5836-4c0a-b6af-439d663afd91 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:38.399586 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47569aa-1a31-4a81-9eab-4cd202ea8ca0 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:38.399586 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e47569aa-1a31-4a81-9eab-4cd202ea8ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:38.419169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47569aa-1a31-4a81-9eab-4cd202ea8ca0 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:38.425945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:38.427671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:38.428160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:38.428652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] PATH_INFO: `/v3/projects/2a154551374b4160bc08095d20f0c49c` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:38.430161 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a154551374b4160bc08095d20f0c49c)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:38.433633 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:38.516571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 874/1718] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:38 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:22:38.517583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e40ae-5836-4c0a-b6af-439d663afd91 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:38.600953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56e40ae-5836-4c0a-b6af-439d663afd91 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:38.611067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3618d-a17f-4a64-80da-46e41e513c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:38.613020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3618d-a17f-4a64-80da-46e41e513c65 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:38.613579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3618d-a17f-4a64-80da-46e41e513c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:38.614048 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3618d-a17f-4a64-80da-46e41e513c65 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:38.616029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3618d-a17f-4a64-80da-46e41e513c65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:38.620312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed3618d-a17f-4a64-80da-46e41e513c65 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:38.665308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a154551374b4160bc08095d20f0c49c', 'request_id': 'req-f7e86551-1347-409a-81d0-5389c8017376'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:38.748334 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 875/1719] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:38 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:38.750014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3618d-a17f-4a64-80da-46e41e513c65 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:38.775645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a154551374b4160bc08095d20f0c49c', 'request_id': 'req-f7e86551-1347-409a-81d0-5389c8017376'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:38.776319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:38.788643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3618d-a17f-4a64-80da-46e41e513c65 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:38.790074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a154551374b4160bc08095d20f0c49c', 'request_id': 'req-f7e86551-1347-409a-81d0-5389c8017376'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:38.809308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] The token cache is being invalidate because project 2a154551374b4160bc08095d20f0c49c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:38.809660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:38.810574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:38.815598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 845/1720] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:38 2022] DELETE /identity/v3/projects/2a154551374b4160bc08095d20f0c49c => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:38.817007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:38.819385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:38.937691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2613f7-cfde-4c83-a71a-f48149d58d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:38.938644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2613f7-cfde-4c83-a71a-f48149d58d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:38.939153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2613f7-cfde-4c83-a71a-f48149d58d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:38.939536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2613f7-cfde-4c83-a71a-f48149d58d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:38.940410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2613f7-cfde-4c83-a71a-f48149d58d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:38.942442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2613f7-cfde-4c83-a71a-f48149d58d06 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:38.955074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 876/1721] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:38.959705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2613f7-cfde-4c83-a71a-f48149d58d06 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:38.961637 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-aa2613f7-cfde-4c83-a71a-f48149d58d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:38.980146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e86551-1347-409a-81d0-5389c8017376 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:38.994533 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:38.995276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:38.995562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:38.995865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] PATH_INFO: `/v3/users/27741abb99384bb7944d4034e211c916` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:38.999092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=27741abb99384bb7944d4034e211c916)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:39.001319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:39.004419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2613f7-cfde-4c83-a71a-f48149d58d06 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:39.017711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de1deb-780a-4da1-a560-646e18f731af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:39.018412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de1deb-780a-4da1-a560-646e18f731af admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:39.018762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de1deb-780a-4da1-a560-646e18f731af admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:39.019109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de1deb-780a-4da1-a560-646e18f731af admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:39.019935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de1deb-780a-4da1-a560-646e18f731af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:39.022347 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de1deb-780a-4da1-a560-646e18f731af admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:39.039949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 877/1722] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:22:39.043640 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de1deb-780a-4da1-a560-646e18f731af admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:39.059231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de1deb-780a-4da1-a560-646e18f731af admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:39.064455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7eecb0-7731-4853-aeba-708ceb7d596f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:39.065185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eecb0-7731-4853-aeba-708ceb7d596f admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:39.065386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eecb0-7731-4853-aeba-708ceb7d596f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:39.065690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eecb0-7731-4853-aeba-708ceb7d596f admin admin] PATH_INFO: `/v3/projects/8f94c57fb8d2487e9650710a859865c0/users/164749f76e654d519433c787e0f76868/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:39.081063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7eecb0-7731-4853-aeba-708ceb7d596f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f94c57fb8d2487e9650710a859865c0, user_id=164749f76e654d519433c787e0f76868, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:39.084390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7eecb0-7731-4853-aeba-708ceb7d596f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:39.187893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 878/1723] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:39 2022] PUT /identity/v3/projects/8f94c57fb8d2487e9650710a859865c0/users/164749f76e654d519433c787e0f76868/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:39.189240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633ed4-3e4c-40aa-95c9-a4654ab5b35b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:39.189530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633ed4-3e4c-40aa-95c9-a4654ab5b35b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:39.189806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633ed4-3e4c-40aa-95c9-a4654ab5b35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:39.267661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-77633ed4-3e4c-40aa-95c9-a4654ab5b35b None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:39.267661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-77633ed4-3e4c-40aa-95c9-a4654ab5b35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:39.285797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27741abb99384bb7944d4034e211c916'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:39.391948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 879/1724] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:22:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:39.394499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27741abb99384bb7944d4034e211c916'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:39.395204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27741abb99384bb7944d4034e211c916'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:39.396011 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27741abb99384bb7944d4034e211c916'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:39.405487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0d4d9-ea59-41dd-b085-f900bbbc7897 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:39.405747 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0d4d9-ea59-41dd-b085-f900bbbc7897 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:39.406037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0d4d9-ea59-41dd-b085-f900bbbc7897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:39.420370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27741abb99384bb7944d4034e211c916'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:39.479710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 846/1725] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:38 2022] DELETE /identity/v3/users/27741abb99384bb7944d4034e211c916 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:39.495877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e7c0d4d9-ea59-41dd-b085-f900bbbc7897 None None] MFA Rules not processed for user `164749f76e654d519433c787e0f76868`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:39.497099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e7c0d4d9-ea59-41dd-b085-f900bbbc7897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:39.625351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 880/1726] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:39.625949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:39.627741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:39.725699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faf94d-21f6-436a-8700-074799177e0f admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:39.741948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb95931-02e1-4571-9f08-4dfe58491c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:39.741948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb95931-02e1-4571-9f08-4dfe58491c70 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:39.742228 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb95931-02e1-4571-9f08-4dfe58491c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:39.742515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb95931-02e1-4571-9f08-4dfe58491c70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:39.743234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb95931-02e1-4571-9f08-4dfe58491c70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:39.747431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb95931-02e1-4571-9f08-4dfe58491c70 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:39.819344 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0d4d9-ea59-41dd-b085-f900bbbc7897 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:39.820840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e7c0d4d9-ea59-41dd-b085-f900bbbc7897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:39.972172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 847/1727] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:39 2022] POST /identity/v3/projects => generated 385 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:22:39.974574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb95931-02e1-4571-9f08-4dfe58491c70 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:40.025007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0d4d9-ea59-41dd-b085-f900bbbc7897 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:40.057656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:40.092315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb95931-02e1-4571-9f08-4dfe58491c70 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:40.097516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48adc5-b719-46f8-9f41-19bbadcabcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:40.098921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48adc5-b719-46f8-9f41-19bbadcabcf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:40.099356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48adc5-b719-46f8-9f41-19bbadcabcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:40.099629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48adc5-b719-46f8-9f41-19bbadcabcf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:40.100365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48adc5-b719-46f8-9f41-19bbadcabcf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:40.102558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48adc5-b719-46f8-9f41-19bbadcabcf5 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:40.139204 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1331b6-0403-4bf0-addd-893885902f8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:40.139994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1331b6-0403-4bf0-addd-893885902f8f service swift] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:40.140342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1331b6-0403-4bf0-addd-893885902f8f service swift] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:40.140691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1331b6-0403-4bf0-addd-893885902f8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:40.143141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9e1331b6-0403-4bf0-addd-893885902f8f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:40.251905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1331b6-0403-4bf0-addd-893885902f8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:40.255897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 848/1728] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:39 2022] POST /identity/v3/users => generated 299 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:22:40.257498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48adc5-b719-46f8-9f41-19bbadcabcf5 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:40.259509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4c48adc5-b719-46f8-9f41-19bbadcabcf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:40.278825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1331b6-0403-4bf0-addd-893885902f8f service swift] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:40.282121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48adc5-b719-46f8-9f41-19bbadcabcf5 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:40.303953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:40.304611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:40.304891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:40.305166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] PATH_INFO: `/v3/projects/31cf563be55b4698ab62606fb8150302` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:40.306338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31cf563be55b4698ab62606fb8150302)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:40.308215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 881/1729] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:22:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 16:22:40.310271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881980ae-167c-4247-a20f-29e4b63e8d23 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:40.310925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881980ae-167c-4247-a20f-29e4b63e8d23 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:40.311153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881980ae-167c-4247-a20f-29e4b63e8d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:40.323152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:40.348227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-881980ae-167c-4247-a20f-29e4b63e8d23 None None] MFA Rules not processed for user `b6a6b52462624e7f84f479144353b51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:40.352954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-881980ae-167c-4247-a20f-29e4b63e8d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:40.480723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 882/1730] 10.222.0.50 () {58 vars in 1015 bytes} [Wed Sep 7 16:22:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:40.482117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881980ae-167c-4247-a20f-29e4b63e8d23 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:40.528258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881980ae-167c-4247-a20f-29e4b63e8d23 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:40.538344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31cf563be55b4698ab62606fb8150302', 'request_id': 'req-41995819-225d-4187-875a-9d6076e38289'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:40.617922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31cf563be55b4698ab62606fb8150302', 'request_id': 'req-41995819-225d-4187-875a-9d6076e38289'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:40.628315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31cf563be55b4698ab62606fb8150302', 'request_id': 'req-41995819-225d-4187-875a-9d6076e38289'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:40.662206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e247199-0e71-46b0-863d-707c7d287516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:40.664863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e247199-0e71-46b0-863d-707c7d287516 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:40.665154 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e247199-0e71-46b0-863d-707c7d287516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:40.665434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e247199-0e71-46b0-863d-707c7d287516 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:40.665821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] The token cache is being invalidate because project 31cf563be55b4698ab62606fb8150302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:40.666124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:40.667830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e247199-0e71-46b0-863d-707c7d287516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:40.672973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e247199-0e71-46b0-863d-707c7d287516 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:40.673474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:40.679509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 849/1731] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:40 2022] DELETE /identity/v3/projects/31cf563be55b4698ab62606fb8150302 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:40.681935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:40.682659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:40.696977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 883/1732] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:40.698698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e247199-0e71-46b0-863d-707c7d287516 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:40.700822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4e247199-0e71-46b0-863d-707c7d287516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:40.725383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41995819-225d-4187-875a-9d6076e38289 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:40.741534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:40.741534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:40.741534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:40.741534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] PATH_INFO: `/v3/users/5df7cf8384e04783a09b571e980b25ea` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:40.744840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df7cf8384e04783a09b571e980b25ea)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:40.748527 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:40.804798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e247199-0e71-46b0-863d-707c7d287516 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:40.819350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3d23c-f167-4265-af93-5061a3267252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:40.819350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3d23c-f167-4265-af93-5061a3267252 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:40.819350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3d23c-f167-4265-af93-5061a3267252 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:40.819350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3d23c-f167-4265-af93-5061a3267252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:40.819734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d1d3d23c-f167-4265-af93-5061a3267252 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:40.823876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3d23c-f167-4265-af93-5061a3267252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:40.825338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3d23c-f167-4265-af93-5061a3267252 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:40.839692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 884/1733] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:40.841660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6653b-8225-4004-a996-e301f56d05b5 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:40.842318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6653b-8225-4004-a996-e301f56d05b5 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:40.842858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6653b-8225-4004-a996-e301f56d05b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:40.916403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-57d6653b-8225-4004-a996-e301f56d05b5 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:40.921809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-57d6653b-8225-4004-a996-e301f56d05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:40.947958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 885/1734] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:40.950364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6653b-8225-4004-a996-e301f56d05b5 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:40.952427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-57d6653b-8225-4004-a996-e301f56d05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:40.997407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6653b-8225-4004-a996-e301f56d05b5 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:41.015105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de70870-a6d5-4586-b4aa-c3aa5cf6d2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:41.016797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de70870-a6d5-4586-b4aa-c3aa5cf6d2ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:41.017301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de70870-a6d5-4586-b4aa-c3aa5cf6d2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:41.017797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de70870-a6d5-4586-b4aa-c3aa5cf6d2ad admin admin] PATH_INFO: `/v3/projects/a3cf9fdc42f042809d00252cc9e53a09/users/1a9be282d08e45a882063b89ff0b8d5b/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:41.056766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df7cf8384e04783a09b571e980b25ea'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:41.070824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de70870-a6d5-4586-b4aa-c3aa5cf6d2ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cf9fdc42f042809d00252cc9e53a09, user_id=1a9be282d08e45a882063b89ff0b8d5b, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:41.073833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de70870-a6d5-4586-b4aa-c3aa5cf6d2ad admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:41.169174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df7cf8384e04783a09b571e980b25ea'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:41.169864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df7cf8384e04783a09b571e980b25ea'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:41.170505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df7cf8384e04783a09b571e980b25ea'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:41.188378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df7cf8384e04783a09b571e980b25ea'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:41.206397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 886/1735] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:40 2022] PUT /identity/v3/projects/a3cf9fdc42f042809d00252cc9e53a09/users/1a9be282d08e45a882063b89ff0b8d5b/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:41.208775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7950b-9455-4c29-8528-d489fd88f1c9 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:41.209132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7950b-9455-4c29-8528-d489fd88f1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:41.209418 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7950b-9455-4c29-8528-d489fd88f1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:41.261246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1aa7950b-9455-4c29-8528-d489fd88f1c9 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:41.262577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1aa7950b-9455-4c29-8528-d489fd88f1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:41.276354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 850/1736] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:40 2022] DELETE /identity/v3/users/5df7cf8384e04783a09b571e980b25ea => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:41.277570 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:41.285421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:41.476518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f30738f-54fc-44cd-a5ae-65308dd41a87 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:41.481421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 887/1737] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:41.484253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be041a-0021-41ea-951e-65c16ade6eb6 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:41.484551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be041a-0021-41ea-951e-65c16ade6eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:41.485676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be041a-0021-41ea-951e-65c16ade6eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:41.494780 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b98f8d-ae09-455c-a3fd-033e241a91bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:41.495745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98f8d-ae09-455c-a3fd-033e241a91bf admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:41.496211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98f8d-ae09-455c-a3fd-033e241a91bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:41.502869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b98f8d-ae09-455c-a3fd-033e241a91bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:41.503134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:41.503207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b98f8d-ae09-455c-a3fd-033e241a91bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:41.503207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b98f8d-ae09-455c-a3fd-033e241a91bf admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:41.511915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 851/1738] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:41.513273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b98f8d-ae09-455c-a3fd-033e241a91bf admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:41.537534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-10be041a-0021-41ea-951e-65c16ade6eb6 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:41.538190 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b98f8d-ae09-455c-a3fd-033e241a91bf admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:41.544847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-10be041a-0021-41ea-951e-65c16ade6eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:41.551360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc5407-f9f7-4705-b1f0-d85128d5463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:41.552614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc5407-f9f7-4705-b1f0-d85128d5463c admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:41.553039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc5407-f9f7-4705-b1f0-d85128d5463c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:41.553603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc5407-f9f7-4705-b1f0-d85128d5463c admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:41.554932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc5407-f9f7-4705-b1f0-d85128d5463c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:41.557278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc5407-f9f7-4705-b1f0-d85128d5463c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:41.558774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 888/1739] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:41.559678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be041a-0021-41ea-951e-65c16ade6eb6 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:41.560494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc5407-f9f7-4705-b1f0-d85128d5463c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:41.600016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be041a-0021-41ea-951e-65c16ade6eb6 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:41.612240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 852/1740] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:41.612240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc5407-f9f7-4705-b1f0-d85128d5463c admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:41.613144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8bbc5407-f9f7-4705-b1f0-d85128d5463c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:41.613532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69203e-a737-41f5-b994-c3afc90531df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:41.614910 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69203e-a737-41f5-b994-c3afc90531df admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:41.615195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69203e-a737-41f5-b994-c3afc90531df admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:41.615472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69203e-a737-41f5-b994-c3afc90531df admin admin] PATH_INFO: `/v3/projects/a3cf9fdc42f042809d00252cc9e53a09/users/1a9be282d08e45a882063b89ff0b8d5b/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:41.622430 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69203e-a737-41f5-b994-c3afc90531df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cf9fdc42f042809d00252cc9e53a09, user_id=1a9be282d08e45a882063b89ff0b8d5b, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:41.646540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69203e-a737-41f5-b994-c3afc90531df admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:41.691593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 889/1741] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:41 2022] PUT /identity/v3/projects/a3cf9fdc42f042809d00252cc9e53a09/users/1a9be282d08e45a882063b89ff0b8d5b/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:41.693554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69203e-a737-41f5-b994-c3afc90531df admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:41.825000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc5407-f9f7-4705-b1f0-d85128d5463c admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:41.825345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69203e-a737-41f5-b994-c3afc90531df admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:41.825345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86ecf6-da39-4ba8-8780-56c425509f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:41.825345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86ecf6-da39-4ba8-8780-56c425509f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:41.825345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86ecf6-da39-4ba8-8780-56c425509f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:41.825624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a86ecf6-da39-4ba8-8780-56c425509f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:41.825624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86ecf6-da39-4ba8-8780-56c425509f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:41.825624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a86ecf6-da39-4ba8-8780-56c425509f29 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:41.851112 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd177261-2acb-49b5-aec2-123c1706d98e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:41.851112 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177261-2acb-49b5-aec2-123c1706d98e service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:41.851112 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177261-2acb-49b5-aec2-123c1706d98e service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:41.861223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177261-2acb-49b5-aec2-123c1706d98e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:41.920475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd177261-2acb-49b5-aec2-123c1706d98e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:41.920475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd177261-2acb-49b5-aec2-123c1706d98e service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:41.920475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 853/1742] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:41.920475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1142f3-97ce-4df1-a199-879831bddea0 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:41.920475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1142f3-97ce-4df1-a199-879831bddea0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:41.920475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1142f3-97ce-4df1-a199-879831bddea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:41.984521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ce1142f3-97ce-4df1-a199-879831bddea0 None None] MFA Rules not processed for user `1a9be282d08e45a882063b89ff0b8d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:41.986781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ce1142f3-97ce-4df1-a199-879831bddea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:42.083412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 890/1743] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:41 2022] POST /identity/v3/projects => generated 357 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:42.087803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86ecf6-da39-4ba8-8780-56c425509f29 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:42.136670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 854/1744] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:42.144840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1142f3-97ce-4df1-a199-879831bddea0 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:42.146629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ce1142f3-97ce-4df1-a199-879831bddea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:42.203445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a86ecf6-da39-4ba8-8780-56c425509f29 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:42.210562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06a4c6-4ec2-4b3a-9b71-ca4000fe1474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:42.211205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06a4c6-4ec2-4b3a-9b71-ca4000fe1474 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:42.211490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06a4c6-4ec2-4b3a-9b71-ca4000fe1474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:42.211824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06a4c6-4ec2-4b3a-9b71-ca4000fe1474 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:42.212431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06a4c6-4ec2-4b3a-9b71-ca4000fe1474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:42.214381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06a4c6-4ec2-4b3a-9b71-ca4000fe1474 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:42.249013 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1142f3-97ce-4df1-a199-879831bddea0 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:42.376392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 891/1745] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:42 2022] POST /identity/v3/users => generated 279 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:22:42.378870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06a4c6-4ec2-4b3a-9b71-ca4000fe1474 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:42.382142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4b06a4c6-4ec2-4b3a-9b71-ca4000fe1474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:42.388080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468ad91-090a-47fa-8fa1-7b40d84eed11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:42.388716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468ad91-090a-47fa-8fa1-7b40d84eed11 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:42.389003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468ad91-090a-47fa-8fa1-7b40d84eed11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:42.389288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468ad91-090a-47fa-8fa1-7b40d84eed11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:42.391663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a468ad91-090a-47fa-8fa1-7b40d84eed11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:42.404267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06a4c6-4ec2-4b3a-9b71-ca4000fe1474 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:42.432212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cfbd92-9f8f-445a-a8f1-6049744b54fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:42.435439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfbd92-9f8f-445a-a8f1-6049744b54fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:42.441630 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfbd92-9f8f-445a-a8f1-6049744b54fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:42.442119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cfbd92-9f8f-445a-a8f1-6049744b54fa admin admin] PATH_INFO: `/v3/users/0fc42e0e1c614ac3a953608eee0e6f08` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:42.449991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfbd92-9f8f-445a-a8f1-6049744b54fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc42e0e1c614ac3a953608eee0e6f08)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:42.455962 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cfbd92-9f8f-445a-a8f1-6049744b54fa admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:42.521646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468ad91-090a-47fa-8fa1-7b40d84eed11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:42.527099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468ad91-090a-47fa-8fa1-7b40d84eed11 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:42.561267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 855/1746] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:42.562946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468ad91-090a-47fa-8fa1-7b40d84eed11 service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:42.579231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468ad91-090a-47fa-8fa1-7b40d84eed11 service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:42.590118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777b7f1-0033-4bf8-985f-7020c89c0d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:42.591598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777b7f1-0033-4bf8-985f-7020c89c0d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:42.592067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777b7f1-0033-4bf8-985f-7020c89c0d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:42.592507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777b7f1-0033-4bf8-985f-7020c89c0d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:42.595931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777b7f1-0033-4bf8-985f-7020c89c0d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:42.595931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777b7f1-0033-4bf8-985f-7020c89c0d4e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:42.611574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 856/1747] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:42.615095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777b7f1-0033-4bf8-985f-7020c89c0d4e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:42.627969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777b7f1-0033-4bf8-985f-7020c89c0d4e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:42.633873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5c072-9447-4f45-97d1-fb5a4d5e4608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:42.634721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5c072-9447-4f45-97d1-fb5a4d5e4608 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:42.635154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5c072-9447-4f45-97d1-fb5a4d5e4608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:42.635575 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5c072-9447-4f45-97d1-fb5a4d5e4608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:42.636424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5c072-9447-4f45-97d1-fb5a4d5e4608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:42.638341 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5c072-9447-4f45-97d1-fb5a4d5e4608 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:42.687347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 857/1748] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:42 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:42.688325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5c072-9447-4f45-97d1-fb5a4d5e4608 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:42.720868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5c072-9447-4f45-97d1-fb5a4d5e4608 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:42.728214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-66cfbd92-9f8f-445a-a8f1-6049744b54fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc42e0e1c614ac3a953608eee0e6f08'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:42.733793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6419d6d-bdde-4160-8d10-c976486195ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:42.739817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6419d6d-bdde-4160-8d10-c976486195ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:42.739817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6419d6d-bdde-4160-8d10-c976486195ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:42.739817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6419d6d-bdde-4160-8d10-c976486195ba admin admin] PATH_INFO: `/v3/projects/fe74614f3bd346b09e90c0a7fb731a92/users/f9db2c80376c4104b3b1d6f04dfd6d65/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:42.752800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6419d6d-bdde-4160-8d10-c976486195ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe74614f3bd346b09e90c0a7fb731a92, user_id=f9db2c80376c4104b3b1d6f04dfd6d65, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:42.756181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6419d6d-bdde-4160-8d10-c976486195ba admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:42.800256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-66cfbd92-9f8f-445a-a8f1-6049744b54fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc42e0e1c614ac3a953608eee0e6f08'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:42.801349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-66cfbd92-9f8f-445a-a8f1-6049744b54fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc42e0e1c614ac3a953608eee0e6f08'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:42.802170 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-66cfbd92-9f8f-445a-a8f1-6049744b54fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc42e0e1c614ac3a953608eee0e6f08'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:42.815586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-66cfbd92-9f8f-445a-a8f1-6049744b54fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc42e0e1c614ac3a953608eee0e6f08'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:42.849764 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 892/1749] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:42 2022] DELETE /identity/v3/users/0fc42e0e1c614ac3a953608eee0e6f08 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:42.856334 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9de97-61ae-4710-a714-13dccad8d4f9 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:42.862210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9de97-61ae-4710-a714-13dccad8d4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:42.862210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9de97-61ae-4710-a714-13dccad8d4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:42.862555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:42.869716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 858/1750] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:42 2022] PUT /identity/v3/projects/fe74614f3bd346b09e90c0a7fb731a92/users/f9db2c80376c4104b3b1d6f04dfd6d65/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:42.887427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d9885-15e9-4b77-8a31-897b38378afb None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:42.887776 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d9885-15e9-4b77-8a31-897b38378afb None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:42.888332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d9885-15e9-4b77-8a31-897b38378afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:42.931902 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-faf9de97-61ae-4710-a714-13dccad8d4f9 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:42.937750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-faf9de97-61ae-4710-a714-13dccad8d4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:42.954313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-496d9885-15e9-4b77-8a31-897b38378afb None None] MFA Rules not processed for user `f9db2c80376c4104b3b1d6f04dfd6d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:42.959606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-496d9885-15e9-4b77-8a31-897b38378afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:43.102253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 893/1751] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:43.103513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050cf37-be60-4ed6-86da-9877d27ef7d7 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:43.104999 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050cf37-be60-4ed6-86da-9877d27ef7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:43.105236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050cf37-be60-4ed6-86da-9877d27ef7d7 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:43.110265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 894/1752] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:22:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:22:43.126567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 859/1753] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:43.131737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050cf37-be60-4ed6-86da-9877d27ef7d7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:43.134379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d050cf37-be60-4ed6-86da-9877d27ef7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:43.155970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d9885-15e9-4b77-8a31-897b38378afb None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:43.157446 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-496d9885-15e9-4b77-8a31-897b38378afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:43.166787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050cf37-be60-4ed6-86da-9877d27ef7d7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:43.187924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fac45-1b78-44b1-99b3-1d1f4e762bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:43.190781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fac45-1b78-44b1-99b3-1d1f4e762bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:43.194235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fac45-1b78-44b1-99b3-1d1f4e762bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:43.196238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fac45-1b78-44b1-99b3-1d1f4e762bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:43.201991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fac45-1b78-44b1-99b3-1d1f4e762bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:43.204504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fac45-1b78-44b1-99b3-1d1f4e762bb0 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:43.273532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d9885-15e9-4b77-8a31-897b38378afb None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:43.282871 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa4e1-a774-4dcd-8969-f88a4c693712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:43.283353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa4e1-a774-4dcd-8969-f88a4c693712 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:43.283638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa4e1-a774-4dcd-8969-f88a4c693712 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:43.283949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3aa4e1-a774-4dcd-8969-f88a4c693712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:43.316180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3aa4e1-a774-4dcd-8969-f88a4c693712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:43.323832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3aa4e1-a774-4dcd-8969-f88a4c693712 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:43.344404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 860/1754] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:43.365644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 895/1755] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:43 2022] POST /identity/v3/projects => generated 355 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:43.367562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa4e1-a774-4dcd-8969-f88a4c693712 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:43.404295 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3aa4e1-a774-4dcd-8969-f88a4c693712 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:43.438383 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fac45-1b78-44b1-99b3-1d1f4e762bb0 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:43.441089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-514fac45-1b78-44b1-99b3-1d1f4e762bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:43.627805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6117e5b-6fc4-41ab-8ed8-c5bd8bec0aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:43.629195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6117e5b-6fc4-41ab-8ed8-c5bd8bec0aee admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:43.630862 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6117e5b-6fc4-41ab-8ed8-c5bd8bec0aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:43.631388 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6117e5b-6fc4-41ab-8ed8-c5bd8bec0aee admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:43.633513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6117e5b-6fc4-41ab-8ed8-c5bd8bec0aee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:43.655441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6117e5b-6fc4-41ab-8ed8-c5bd8bec0aee admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:43.705037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fac45-1b78-44b1-99b3-1d1f4e762bb0 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:43.717478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:43.718434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:43.718733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:43.719065 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] PATH_INFO: `/v3/projects/b314ea662c5148abb80ea87a3e77cd19` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:43.726938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b314ea662c5148abb80ea87a3e77cd19)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:43.729140 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:43.783845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 861/1756] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:43 2022] POST /identity/v3/users => generated 284 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:43.785252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256fc36-490a-4883-b930-c1f4f2a78b03 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:43.785624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256fc36-490a-4883-b930-c1f4f2a78b03 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:43.786949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256fc36-490a-4883-b930-c1f4f2a78b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:43.839543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c256fc36-490a-4883-b930-c1f4f2a78b03 None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:43.841039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c256fc36-490a-4883-b930-c1f4f2a78b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:43.868289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:43.917918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b314ea662c5148abb80ea87a3e77cd19', 'request_id': 'req-92baefe9-e41e-4591-9b3e-dea422854471'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:43.955304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b314ea662c5148abb80ea87a3e77cd19', 'request_id': 'req-92baefe9-e41e-4591-9b3e-dea422854471'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:44.000391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 862/1757] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:22:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:44.001285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256fc36-490a-4883-b930-c1f4f2a78b03 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:44.018554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256fc36-490a-4883-b930-c1f4f2a78b03 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:44.058055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b314ea662c5148abb80ea87a3e77cd19', 'request_id': 'req-92baefe9-e41e-4591-9b3e-dea422854471'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:44.109826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] The token cache is being invalidate because project b314ea662c5148abb80ea87a3e77cd19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:44.110128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:44.113193 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:44.123927 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 896/1758] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:43 2022] DELETE /identity/v3/projects/b314ea662c5148abb80ea87a3e77cd19 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:22:44.125399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:44.130245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:44.180466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d561bd-3ffc-4af4-a049-ec810f06132d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:44.181135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d561bd-3ffc-4af4-a049-ec810f06132d admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:44.181418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d561bd-3ffc-4af4-a049-ec810f06132d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:44.181692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d561bd-3ffc-4af4-a049-ec810f06132d admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:44.182348 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d561bd-3ffc-4af4-a049-ec810f06132d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:44.184314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d561bd-3ffc-4af4-a049-ec810f06132d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:44.209797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 863/1759] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:44.217047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99757-497c-4b03-bfe9-ccd4884da341 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:44.217047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99757-497c-4b03-bfe9-ccd4884da341 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:44.217047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f99757-497c-4b03-bfe9-ccd4884da341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:44.267104 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-d8f99757-497c-4b03-bfe9-ccd4884da341 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:44.268419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d8f99757-497c-4b03-bfe9-ccd4884da341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:44.289854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 864/1760] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:44.290915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f99757-497c-4b03-bfe9-ccd4884da341 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:44.292136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d8f99757-497c-4b03-bfe9-ccd4884da341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:44.317693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f99757-497c-4b03-bfe9-ccd4884da341 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:44.331136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a700d1-eaaa-4feb-8a56-a8906b21d8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:44.331895 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a700d1-eaaa-4feb-8a56-a8906b21d8f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:44.332175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a700d1-eaaa-4feb-8a56-a8906b21d8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:44.332556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a700d1-eaaa-4feb-8a56-a8906b21d8f9 admin admin] PATH_INFO: `/v3/projects/c75e48ae033d42679c98b796138a08a7/users/2ac809ede871453c9d8371d92b779048/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:44.370520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a700d1-eaaa-4feb-8a56-a8906b21d8f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75e48ae033d42679c98b796138a08a7, user_id=2ac809ede871453c9d8371d92b779048, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:44.373305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a700d1-eaaa-4feb-8a56-a8906b21d8f9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:44.408482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92baefe9-e41e-4591-9b3e-dea422854471 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:44.414836 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93b591-66c6-4841-90bd-3e20f6469905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:44.415661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93b591-66c6-4841-90bd-3e20f6469905 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:44.416360 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93b591-66c6-4841-90bd-3e20f6469905 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:44.416692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93b591-66c6-4841-90bd-3e20f6469905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:44.419174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9b93b591-66c6-4841-90bd-3e20f6469905 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:44.457186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93b591-66c6-4841-90bd-3e20f6469905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:44.472732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93b591-66c6-4841-90bd-3e20f6469905 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:44.485502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 897/1761] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:44.493126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884feba6-215e-45fe-ad36-03239c772e1f None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:44.495573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884feba6-215e-45fe-ad36-03239c772e1f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:44.496128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884feba6-215e-45fe-ad36-03239c772e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:44.505670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 865/1762] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:44 2022] PUT /identity/v3/projects/c75e48ae033d42679c98b796138a08a7/users/2ac809ede871453c9d8371d92b779048/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:44.509667 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a700d1-eaaa-4feb-8a56-a8906b21d8f9 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:44.546346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-884feba6-215e-45fe-ad36-03239c772e1f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:44.547992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-884feba6-215e-45fe-ad36-03239c772e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:44.616605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=79,keystone:INSERT=6,keystone:DELETE=15 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:44.695054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 898/1763] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:44.698422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884feba6-215e-45fe-ad36-03239c772e1f None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:44.704009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-884feba6-215e-45fe-ad36-03239c772e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:44.706986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a700d1-eaaa-4feb-8a56-a8906b21d8f9 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:44.731605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884feba6-215e-45fe-ad36-03239c772e1f None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:44.735164 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c8ff0-eea8-428a-ab46-f841ab4e4236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:44.737718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c8ff0-eea8-428a-ab46-f841ab4e4236 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:44.738008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c8ff0-eea8-428a-ab46-f841ab4e4236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:44.739550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:44.740725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:44.741164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:44.741542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] PATH_INFO: `/v3/projects/9b4946c885204c5592a96bbeefde3ff1` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:44.742444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c8ff0-eea8-428a-ab46-f841ab4e4236 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:44.742959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b4946c885204c5592a96bbeefde3ff1)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:44.744687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c8ff0-eea8-428a-ab46-f841ab4e4236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:44.745281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:44.788349 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c8ff0-eea8-428a-ab46-f841ab4e4236 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:44.828547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 866/1764] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:44.828895 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c8ff0-eea8-428a-ab46-f841ab4e4236 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:44.856813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c8ff0-eea8-428a-ab46-f841ab4e4236 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:44.896321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4946c885204c5592a96bbeefde3ff1', 'request_id': 'req-697a09f6-fc34-45c6-aefa-72c7b463340e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:44.907732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4946c885204c5592a96bbeefde3ff1', 'request_id': 'req-697a09f6-fc34-45c6-aefa-72c7b463340e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:45.013387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4946c885204c5592a96bbeefde3ff1', 'request_id': 'req-697a09f6-fc34-45c6-aefa-72c7b463340e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:45.033233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] The token cache is being invalidate because project 9b4946c885204c5592a96bbeefde3ff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:45.037644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:45.037940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:45.040265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 899/1765] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:44 2022] DELETE /identity/v3/projects/9b4946c885204c5592a96bbeefde3ff1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:45.041626 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:45.044275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:45.057450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9aaf12-317b-4b06-a56b-1450a265ffdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:45.058741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9aaf12-317b-4b06-a56b-1450a265ffdb admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:45.060386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9aaf12-317b-4b06-a56b-1450a265ffdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:45.060693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9aaf12-317b-4b06-a56b-1450a265ffdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:45.061382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9aaf12-317b-4b06-a56b-1450a265ffdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:45.061699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9aaf12-317b-4b06-a56b-1450a265ffdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:45.067231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9aaf12-317b-4b06-a56b-1450a265ffdb admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:45.082575 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a09f6-fc34-45c6-aefa-72c7b463340e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:45.086354 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:45.092554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80ea83-961c-45ae-ba72-855cc0dd9333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:45.093560 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80ea83-961c-45ae-ba72-855cc0dd9333 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:45.094137 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80ea83-961c-45ae-ba72-855cc0dd9333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:45.094735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80ea83-961c-45ae-ba72-855cc0dd9333 admin admin] PATH_INFO: `/v3/projects/c75e48ae033d42679c98b796138a08a7/users/2ac809ede871453c9d8371d92b779048/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:45.101338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 867/1766] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:45.101338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9aaf12-317b-4b06-a56b-1450a265ffdb admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:45.102745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80ea83-961c-45ae-ba72-855cc0dd9333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c75e48ae033d42679c98b796138a08a7, user_id=2ac809ede871453c9d8371d92b779048, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:45.103375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4f9aaf12-317b-4b06-a56b-1450a265ffdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:45.109402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80ea83-961c-45ae-ba72-855cc0dd9333 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:45.139457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9aaf12-317b-4b06-a56b-1450a265ffdb admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:45.145797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 900/1767] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:45 2022] PUT /identity/v3/projects/c75e48ae033d42679c98b796138a08a7/users/2ac809ede871453c9d8371d92b779048/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:45.147004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80ea83-961c-45ae-ba72-855cc0dd9333 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:45.154173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4f80ea83-961c-45ae-ba72-855cc0dd9333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:45.278544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:45.279205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:45.279498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:45.279805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] PATH_INFO: `/v3/users/7fd31f1e4f3d440c9227eb69d9cd5ee3` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:45.281782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd31f1e4f3d440c9227eb69d9cd5ee3)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:45.283979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:45.333346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80ea83-961c-45ae-ba72-855cc0dd9333 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:45.345668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61bd52-51c8-4681-a785-a51e02a29003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:45.346249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61bd52-51c8-4681-a785-a51e02a29003 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:45.348896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61bd52-51c8-4681-a785-a51e02a29003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:45.349187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61bd52-51c8-4681-a785-a51e02a29003 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:45.350770 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61bd52-51c8-4681-a785-a51e02a29003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:45.357840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61bd52-51c8-4681-a785-a51e02a29003 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:45.509383 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 901/1768] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:45 2022] POST /identity/v3/projects => generated 367 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:45.511072 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcc499-c3a9-4af5-ab2f-9d0b5827fd87 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:45.511462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcc499-c3a9-4af5-ab2f-9d0b5827fd87 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:45.512199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcc499-c3a9-4af5-ab2f-9d0b5827fd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:45.559384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ebbcc499-c3a9-4af5-ab2f-9d0b5827fd87 None None] MFA Rules not processed for user `2ac809ede871453c9d8371d92b779048`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:45.564634 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ebbcc499-c3a9-4af5-ab2f-9d0b5827fd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:45.664381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd31f1e4f3d440c9227eb69d9cd5ee3'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:45.699517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:45.732164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 902/1769] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:45.736997 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcc499-c3a9-4af5-ab2f-9d0b5827fd87 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:45.773613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd31f1e4f3d440c9227eb69d9cd5ee3'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:45.773613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd31f1e4f3d440c9227eb69d9cd5ee3'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:45.773613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd31f1e4f3d440c9227eb69d9cd5ee3'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:45.784230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd31f1e4f3d440c9227eb69d9cd5ee3'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:45.826566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 868/1770] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:45 2022] DELETE /identity/v3/users/7fd31f1e4f3d440c9227eb69d9cd5ee3 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:45.827701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:45.829686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:45.943733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749de53b-5187-4784-8ca9-d2f354c89c7f admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:45.946385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbcc499-c3a9-4af5-ab2f-9d0b5827fd87 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:45.948529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadb582-ed96-42c7-87f2-02901a9d8fdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:45.949365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadb582-ed96-42c7-87f2-02901a9d8fdd service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:45.949856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadb582-ed96-42c7-87f2-02901a9d8fdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:45.950625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aadb582-ed96-42c7-87f2-02901a9d8fdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:45.953413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7aadb582-ed96-42c7-87f2-02901a9d8fdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:46.040993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadb582-ed96-42c7-87f2-02901a9d8fdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:46.046093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aadb582-ed96-42c7-87f2-02901a9d8fdd service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:46.064589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 869/1771] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:45 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:46.115169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:46.162030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f1b83-d906-45fb-a466-10e377376988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:46.163025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f1b83-d906-45fb-a466-10e377376988 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:46.163752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f1b83-d906-45fb-a466-10e377376988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:46.166568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:46.167700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f1b83-d906-45fb-a466-10e377376988 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:46.180502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f1b83-d906-45fb-a466-10e377376988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:46.182621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f1b83-d906-45fb-a466-10e377376988 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:46.319603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 903/1772] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:45 2022] POST /identity/v3/users => generated 284 bytes in 577 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:46.319732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadb582-ed96-42c7-87f2-02901a9d8fdd service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:46.319732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7aadb582-ed96-42c7-87f2-02901a9d8fdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:46.340099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aadb582-ed96-42c7-87f2-02901a9d8fdd service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:46.358618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449512a-879d-4de9-b500-8fe34f944d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:46.360254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449512a-879d-4de9-b500-8fe34f944d28 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:46.360701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449512a-879d-4de9-b500-8fe34f944d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:46.361114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449512a-879d-4de9-b500-8fe34f944d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:46.362014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449512a-879d-4de9-b500-8fe34f944d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:46.368931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449512a-879d-4de9-b500-8fe34f944d28 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:46.385786 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 870/1773] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:46.389460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f1b83-d906-45fb-a466-10e377376988 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:46.401942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f1b83-d906-45fb-a466-10e377376988 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:46.408484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd659-114b-49fa-bb44-ecacb74b9abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:46.409131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd659-114b-49fa-bb44-ecacb74b9abf admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:46.409416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd659-114b-49fa-bb44-ecacb74b9abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:46.409783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10cd659-114b-49fa-bb44-ecacb74b9abf admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:46.410882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cd659-114b-49fa-bb44-ecacb74b9abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:46.412693 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10cd659-114b-49fa-bb44-ecacb74b9abf admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:46.434055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 904/1774] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:46.436125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449512a-879d-4de9-b500-8fe34f944d28 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:46.448675 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449512a-879d-4de9-b500-8fe34f944d28 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:46.483934 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1dc0a9-5b21-4ef3-b222-e69548abc8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:46.484721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dc0a9-5b21-4ef3-b222-e69548abc8fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:46.485007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dc0a9-5b21-4ef3-b222-e69548abc8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:46.485296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dc0a9-5b21-4ef3-b222-e69548abc8fb admin admin] PATH_INFO: `/v3/projects/af4a76490ae847c3a6d679f68f1d6582/users/d2f6dafa676144cfb3e501c72860e50c/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:46.542218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1dc0a9-5b21-4ef3-b222-e69548abc8fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=af4a76490ae847c3a6d679f68f1d6582, user_id=d2f6dafa676144cfb3e501c72860e50c, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:46.545077 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1dc0a9-5b21-4ef3-b222-e69548abc8fb admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:46.625859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd659-114b-49fa-bb44-ecacb74b9abf admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:46.637470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10cd659-114b-49fa-bb44-ecacb74b9abf admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:46.641622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562a2aa-b577-4ed5-8c18-2814b6486438 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:46.642527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a2aa-b577-4ed5-8c18-2814b6486438 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:46.642879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a2aa-b577-4ed5-8c18-2814b6486438 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:46.643232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562a2aa-b577-4ed5-8c18-2814b6486438 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:46.645570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8562a2aa-b577-4ed5-8c18-2814b6486438 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:46.649846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 871/1775] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:46 2022] PUT /identity/v3/projects/af4a76490ae847c3a6d679f68f1d6582/users/d2f6dafa676144cfb3e501c72860e50c/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:46.664121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545afd93-b3d1-4919-9b3b-8da38cee3078 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:46.664409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545afd93-b3d1-4919-9b3b-8da38cee3078 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:46.664685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545afd93-b3d1-4919-9b3b-8da38cee3078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:46.762258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-545afd93-b3d1-4919-9b3b-8da38cee3078 None None] MFA Rules not processed for user `d2f6dafa676144cfb3e501c72860e50c`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:46.763644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-545afd93-b3d1-4919-9b3b-8da38cee3078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:46.771546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562a2aa-b577-4ed5-8c18-2814b6486438 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:46.773706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562a2aa-b577-4ed5-8c18-2814b6486438 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:46.796414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 905/1776] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:46.917579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 872/1777] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:46.922758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562a2aa-b577-4ed5-8c18-2814b6486438 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:46.949975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562a2aa-b577-4ed5-8c18-2814b6486438 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:47.074289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c22e90-1f68-4768-8eeb-d9b14ee03338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:47.074995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c22e90-1f68-4768-8eeb-d9b14ee03338 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:47.075277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c22e90-1f68-4768-8eeb-d9b14ee03338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:47.075636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c22e90-1f68-4768-8eeb-d9b14ee03338 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:47.076304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c22e90-1f68-4768-8eeb-d9b14ee03338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:47.078483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c22e90-1f68-4768-8eeb-d9b14ee03338 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:47.112229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:47.198781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:47.215427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 906/1778] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:46 2022] POST /identity/v3/projects => generated 369 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:47.219840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545afd93-b3d1-4919-9b3b-8da38cee3078 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:47.238927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545afd93-b3d1-4919-9b3b-8da38cee3078 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:47.369857 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c22e90-1f68-4768-8eeb-d9b14ee03338 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:47.373773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b1c22e90-1f68-4768-8eeb-d9b14ee03338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:47.410258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb3501-dda3-4924-bd9d-3d5b3c268edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:47.411322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb3501-dda3-4924-bd9d-3d5b3c268edb admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:47.411615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb3501-dda3-4924-bd9d-3d5b3c268edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:47.412529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb3501-dda3-4924-bd9d-3d5b3c268edb admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:47.413207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb3501-dda3-4924-bd9d-3d5b3c268edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:47.415943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb3501-dda3-4924-bd9d-3d5b3c268edb admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:47.516807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c22e90-1f68-4768-8eeb-d9b14ee03338 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:47.524097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:47.525028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:47.525442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:47.525902 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] PATH_INFO: `/v3/projects/ca5d9692a8c24d49925e1935ae2cd929` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:47.527258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca5d9692a8c24d49925e1935ae2cd929)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:47.530039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:47.583859 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 873/1779] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:47 2022] POST /identity/v3/users => generated 291 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:47.588769 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047f716-b458-433f-869c-788445cd9a36 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:47.588769 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047f716-b458-433f-869c-788445cd9a36 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:47.588769 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1047f716-b458-433f-869c-788445cd9a36 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:47.588769 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 874/1780] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:22:47.588769 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047f716-b458-433f-869c-788445cd9a36 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:47.603316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1047f716-b458-433f-869c-788445cd9a36 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:47.608747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38e059-0434-42ea-b261-3504efe9a70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:47.609435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38e059-0434-42ea-b261-3504efe9a70a admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:47.609933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38e059-0434-42ea-b261-3504efe9a70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:47.610022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38e059-0434-42ea-b261-3504efe9a70a admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:47.610627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38e059-0434-42ea-b261-3504efe9a70a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:47.612590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38e059-0434-42ea-b261-3504efe9a70a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:47.635660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 875/1781] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:47.639357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e75924-ba3b-4834-be6e-9f3184e7dbc9 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:47.639357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e75924-ba3b-4834-be6e-9f3184e7dbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:47.639357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e75924-ba3b-4834-be6e-9f3184e7dbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:47.689606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-67e75924-ba3b-4834-be6e-9f3184e7dbc9 None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:47.698080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-67e75924-ba3b-4834-be6e-9f3184e7dbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:47.744505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca5d9692a8c24d49925e1935ae2cd929', 'request_id': 'req-40c997ec-70ab-4c2c-843c-66e158e3d5d6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:47.758968 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca5d9692a8c24d49925e1935ae2cd929', 'request_id': 'req-40c997ec-70ab-4c2c-843c-66e158e3d5d6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:47.872288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 876/1782] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:22:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:47.873493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e75924-ba3b-4834-be6e-9f3184e7dbc9 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:47.889215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e75924-ba3b-4834-be6e-9f3184e7dbc9 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:47.913181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca5d9692a8c24d49925e1935ae2cd929', 'request_id': 'req-40c997ec-70ab-4c2c-843c-66e158e3d5d6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:47.936278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] The token cache is being invalidate because project ca5d9692a8c24d49925e1935ae2cd929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:47.936575 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:47.939174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-40c997ec-70ab-4c2c-843c-66e158e3d5d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:47.941043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 907/1783] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:47 2022] DELETE /identity/v3/projects/ca5d9692a8c24d49925e1935ae2cd929 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:47.948005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a2e8-73cd-47ae-a402-8c21706ceed9 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:47.948517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a2e8-73cd-47ae-a402-8c21706ceed9 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:47.949041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a2e8-73cd-47ae-a402-8c21706ceed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:47.978084 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:47.991384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-d983a2e8-73cd-47ae-a402-8c21706ceed9 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:47.992878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d983a2e8-73cd-47ae-a402-8c21706ceed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:48.031044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d503c-0de8-4da0-9615-7f2e2c09e6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:48.031947 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d503c-0de8-4da0-9615-7f2e2c09e6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:48.032227 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d503c-0de8-4da0-9615-7f2e2c09e6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:48.032540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d503c-0de8-4da0-9615-7f2e2c09e6e9 admin admin] PATH_INFO: `/v3/projects/e9021af8696540108d6decc797c966fa/users/83432b147a30487cbcb84b2940a6df41/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:48.063223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d503c-0de8-4da0-9615-7f2e2c09e6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9021af8696540108d6decc797c966fa, user_id=83432b147a30487cbcb84b2940a6df41, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:48.065991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d503c-0de8-4da0-9615-7f2e2c09e6e9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:48.147250 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 908/1784] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:48.153753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983a2e8-73cd-47ae-a402-8c21706ceed9 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:48.153753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d983a2e8-73cd-47ae-a402-8c21706ceed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:48.184763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 877/1785] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:47 2022] PUT /identity/v3/projects/e9021af8696540108d6decc797c966fa/users/83432b147a30487cbcb84b2940a6df41/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:48.189029 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061856c-884f-46f6-b9fd-d8794f6bf97c None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:48.190418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061856c-884f-46f6-b9fd-d8794f6bf97c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:48.190771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8061856c-884f-46f6-b9fd-d8794f6bf97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:48.266548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-8061856c-884f-46f6-b9fd-d8794f6bf97c None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:48.276367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8061856c-884f-46f6-b9fd-d8794f6bf97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:48.295663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983a2e8-73cd-47ae-a402-8c21706ceed9 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:48.443111 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 878/1786] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:48.446129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061856c-884f-46f6-b9fd-d8794f6bf97c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:48.463142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8061856c-884f-46f6-b9fd-d8794f6bf97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:48.479000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8061856c-884f-46f6-b9fd-d8794f6bf97c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:48.483904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc64d7-d42a-4f40-acc3-316163363736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:48.485248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc64d7-d42a-4f40-acc3-316163363736 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:48.485542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc64d7-d42a-4f40-acc3-316163363736 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:48.485837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc64d7-d42a-4f40-acc3-316163363736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:48.489799 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-24fc64d7-d42a-4f40-acc3-316163363736 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:48.493753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf695ae-71e9-4971-b5f8-200a8c95ea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:48.494440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf695ae-71e9-4971-b5f8-200a8c95ea07 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:48.494725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf695ae-71e9-4971-b5f8-200a8c95ea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:48.499275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf695ae-71e9-4971-b5f8-200a8c95ea07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:48.499275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf695ae-71e9-4971-b5f8-200a8c95ea07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:48.499275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf695ae-71e9-4971-b5f8-200a8c95ea07 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:48.515721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 879/1787] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:48.524140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf695ae-71e9-4971-b5f8-200a8c95ea07 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:48.533801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc64d7-d42a-4f40-acc3-316163363736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:48.537058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc64d7-d42a-4f40-acc3-316163363736 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:48.542785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf695ae-71e9-4971-b5f8-200a8c95ea07 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:48.549952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f879c-3b56-456c-b7ff-ec6191565592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:48.550628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f879c-3b56-456c-b7ff-ec6191565592 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:48.551661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f879c-3b56-456c-b7ff-ec6191565592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:48.551993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f879c-3b56-456c-b7ff-ec6191565592 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:48.553238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 909/1788] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:48.554146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc64d7-d42a-4f40-acc3-316163363736 service nova] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:48.560330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f879c-3b56-456c-b7ff-ec6191565592 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:48.560330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f879c-3b56-456c-b7ff-ec6191565592 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:48.560330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f879c-3b56-456c-b7ff-ec6191565592 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:48.571803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc64d7-d42a-4f40-acc3-316163363736 service nova] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:48.584606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 880/1789] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:48.589914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f879c-3b56-456c-b7ff-ec6191565592 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:48.590255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cb2dc-5b0b-4fb0-9279-b4a295f19b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:48.590934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cb2dc-5b0b-4fb0-9279-b4a295f19b7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:48.591268 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cb2dc-5b0b-4fb0-9279-b4a295f19b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:48.591492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cb2dc-5b0b-4fb0-9279-b4a295f19b7b admin admin] PATH_INFO: `/v3/projects/e9021af8696540108d6decc797c966fa/users/83432b147a30487cbcb84b2940a6df41/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:48.593639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cb2dc-5b0b-4fb0-9279-b4a295f19b7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9021af8696540108d6decc797c966fa, user_id=83432b147a30487cbcb84b2940a6df41, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:48.597391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cb2dc-5b0b-4fb0-9279-b4a295f19b7b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:48.607181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f879c-3b56-456c-b7ff-ec6191565592 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:48.612971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef8063-3b60-4f9e-b19f-ad5c54f57881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:48.614697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef8063-3b60-4f9e-b19f-ad5c54f57881 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:48.618144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef8063-3b60-4f9e-b19f-ad5c54f57881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:48.618144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef8063-3b60-4f9e-b19f-ad5c54f57881 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:48.618144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef8063-3b60-4f9e-b19f-ad5c54f57881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:48.619211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef8063-3b60-4f9e-b19f-ad5c54f57881 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:48.641335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 910/1790] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:48 2022] PUT /identity/v3/projects/e9021af8696540108d6decc797c966fa/users/83432b147a30487cbcb84b2940a6df41/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:48.646790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2286c6-11fc-4fcc-a420-da1f6e26a67c None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:48.646790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2286c6-11fc-4fcc-a420-da1f6e26a67c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:48.646790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2286c6-11fc-4fcc-a420-da1f6e26a67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:48.709855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8a2286c6-11fc-4fcc-a420-da1f6e26a67c None None] MFA Rules not processed for user `83432b147a30487cbcb84b2940a6df41`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:48.711837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8a2286c6-11fc-4fcc-a420-da1f6e26a67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:48.759264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 881/1791] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:48 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:48.769940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef8063-3b60-4f9e-b19f-ad5c54f57881 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:48.810910 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 911/1792] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:48.940539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2286c6-11fc-4fcc-a420-da1f6e26a67c None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:48.944169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8a2286c6-11fc-4fcc-a420-da1f6e26a67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:48.963520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef8063-3b60-4f9e-b19f-ad5c54f57881 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:48.975190 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e65b3d-8690-4f8e-87fb-ce3dcf36045d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:48.975915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e65b3d-8690-4f8e-87fb-ce3dcf36045d admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:48.976484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e65b3d-8690-4f8e-87fb-ce3dcf36045d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:48.976484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e65b3d-8690-4f8e-87fb-ce3dcf36045d admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:48.979930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e65b3d-8690-4f8e-87fb-ce3dcf36045d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:48.984497 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e65b3d-8690-4f8e-87fb-ce3dcf36045d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:49.052096 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2286c6-11fc-4fcc-a420-da1f6e26a67c None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:49.124742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 882/1793] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:48 2022] POST /identity/v3/users => generated 294 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:49.129296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e65b3d-8690-4f8e-87fb-ce3dcf36045d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:49.150414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e65b3d-8690-4f8e-87fb-ce3dcf36045d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:49.156684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa655c0-3eec-4a1e-ba26-d82f48e16027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:49.157288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa655c0-3eec-4a1e-ba26-d82f48e16027 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:49.157797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa655c0-3eec-4a1e-ba26-d82f48e16027 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:49.157961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa655c0-3eec-4a1e-ba26-d82f48e16027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:49.166356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81e612-9d0e-4b1b-b6af-e1c5fa7fed81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:49.167197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81e612-9d0e-4b1b-b6af-e1c5fa7fed81 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:49.167483 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81e612-9d0e-4b1b-b6af-e1c5fa7fed81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:49.167782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81e612-9d0e-4b1b-b6af-e1c5fa7fed81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:49.169627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81e612-9d0e-4b1b-b6af-e1c5fa7fed81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:49.171481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81e612-9d0e-4b1b-b6af-e1c5fa7fed81 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:49.200479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 883/1794] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:49.201987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81e612-9d0e-4b1b-b6af-e1c5fa7fed81 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:49.221042 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81e612-9d0e-4b1b-b6af-e1c5fa7fed81 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:49.228150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862fc9b-6e8c-47d6-a671-98dd6f504788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:49.228150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862fc9b-6e8c-47d6-a671-98dd6f504788 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:49.228444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862fc9b-6e8c-47d6-a671-98dd6f504788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:49.228737 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862fc9b-6e8c-47d6-a671-98dd6f504788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:49.229493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862fc9b-6e8c-47d6-a671-98dd6f504788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:49.231424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862fc9b-6e8c-47d6-a671-98dd6f504788 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:49.245341 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 884/1795] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:49.250321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862fc9b-6e8c-47d6-a671-98dd6f504788 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:49.262854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862fc9b-6e8c-47d6-a671-98dd6f504788 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:49.273735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b3709-746e-40be-914d-1e179d45da8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:49.274715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b3709-746e-40be-914d-1e179d45da8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:49.275004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b3709-746e-40be-914d-1e179d45da8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:49.275474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b3709-746e-40be-914d-1e179d45da8e admin admin] PATH_INFO: `/v3/projects/0e6ee18d803549898d6905a26c67f422/users/047dea136c4b48dd999c4998b8af4963/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:49.290068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa655c0-3eec-4a1e-ba26-d82f48e16027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:49.297857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b3709-746e-40be-914d-1e179d45da8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e6ee18d803549898d6905a26c67f422, user_id=047dea136c4b48dd999c4998b8af4963, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:49.300289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa655c0-3eec-4a1e-ba26-d82f48e16027 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:49.301666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b3709-746e-40be-914d-1e179d45da8e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:49.319248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 912/1796] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:49.325202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:49.385057 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:49.401195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 885/1797] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:49 2022] PUT /identity/v3/projects/0e6ee18d803549898d6905a26c67f422/users/047dea136c4b48dd999c4998b8af4963/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:49.402640 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f4035-8a50-4df5-8f6b-f7b8f236bb63 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:49.403125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f4035-8a50-4df5-8f6b-f7b8f236bb63 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:49.403416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f4035-8a50-4df5-8f6b-f7b8f236bb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:49.443495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-0b1f4035-8a50-4df5-8f6b-f7b8f236bb63 None None] MFA Rules not processed for user `047dea136c4b48dd999c4998b8af4963`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:49.448713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0b1f4035-8a50-4df5-8f6b-f7b8f236bb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:49.551388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 913/1798] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:49.552935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b3709-746e-40be-914d-1e179d45da8e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:49.657541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b3709-746e-40be-914d-1e179d45da8e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:49.678612 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9d423-b20c-44ee-b3b6-9455fc10f573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:49.679814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9d423-b20c-44ee-b3b6-9455fc10f573 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:49.680305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9d423-b20c-44ee-b3b6-9455fc10f573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:49.680765 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9d423-b20c-44ee-b3b6-9455fc10f573 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:49.681792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9d423-b20c-44ee-b3b6-9455fc10f573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:49.684394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9d423-b20c-44ee-b3b6-9455fc10f573 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:49.784117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 886/1799] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:49 2022] POST /identity/v3/projects => generated 385 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:49.788680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f4035-8a50-4df5-8f6b-f7b8f236bb63 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:49.803880 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f4035-8a50-4df5-8f6b-f7b8f236bb63 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:49.880689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e5fa-cf63-4396-a610-3c0049db0e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:49.881312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e5fa-cf63-4396-a610-3c0049db0e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:49.882977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e5fa-cf63-4396-a610-3c0049db0e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:49.882977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e5fa-cf63-4396-a610-3c0049db0e48 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:49.882977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3e5fa-cf63-4396-a610-3c0049db0e48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:49.884560 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3e5fa-cf63-4396-a610-3c0049db0e48 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:50.064290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 914/1800] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:49 2022] POST /identity/v3/users => generated 299 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:50.069744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9d423-b20c-44ee-b3b6-9455fc10f573 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:50.084847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9d423-b20c-44ee-b3b6-9455fc10f573 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:50.098120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139922d2-75ae-42e8-9005-d88e8fa6d0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:50.098120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139922d2-75ae-42e8-9005-d88e8fa6d0ad admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:50.098120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139922d2-75ae-42e8-9005-d88e8fa6d0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:50.098120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139922d2-75ae-42e8-9005-d88e8fa6d0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:50.098377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139922d2-75ae-42e8-9005-d88e8fa6d0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:50.098377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139922d2-75ae-42e8-9005-d88e8fa6d0ad admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:50.122686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e5fa-cf63-4396-a610-3c0049db0e48 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:50.123114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 887/1801] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:50.139153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e5fa-cf63-4396-a610-3c0049db0e48 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:50.152807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d9c20-5616-4211-968b-dc0137204014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:50.153561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d9c20-5616-4211-968b-dc0137204014 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:50.154467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d9c20-5616-4211-968b-dc0137204014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:50.154754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d9c20-5616-4211-968b-dc0137204014 admin admin] PATH_INFO: `/v3/projects/6ab6e04f13a347a6bc0dc1099ee8d3f4/users/78ea77f42eab456e8e5ba45228f19106/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:50.189506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d9c20-5616-4211-968b-dc0137204014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ab6e04f13a347a6bc0dc1099ee8d3f4, user_id=78ea77f42eab456e8e5ba45228f19106, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:50.195708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d9c20-5616-4211-968b-dc0137204014 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:50.261535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 915/1802] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:50 2022] PUT /identity/v3/projects/6ab6e04f13a347a6bc0dc1099ee8d3f4/users/78ea77f42eab456e8e5ba45228f19106/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:50.265524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139922d2-75ae-42e8-9005-d88e8fa6d0ad admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:50.285906 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139922d2-75ae-42e8-9005-d88e8fa6d0ad admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:50.393491 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88385d-42a6-4764-9204-89d18d411e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:50.394028 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88385d-42a6-4764-9204-89d18d411e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:50.394310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88385d-42a6-4764-9204-89d18d411e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:50.394621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88385d-42a6-4764-9204-89d18d411e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:50.395270 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88385d-42a6-4764-9204-89d18d411e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:50.396885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88385d-42a6-4764-9204-89d18d411e80 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:50.407822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 888/1803] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:50.412295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d9c20-5616-4211-968b-dc0137204014 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:50.424057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d9c20-5616-4211-968b-dc0137204014 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:50.430226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6dbab-d0d8-483a-8ac5-3f190a324a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:50.430905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6dbab-d0d8-483a-8ac5-3f190a324a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:50.431221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6dbab-d0d8-483a-8ac5-3f190a324a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:50.433697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6dbab-d0d8-483a-8ac5-3f190a324a19 admin admin] PATH_INFO: `/v3/projects/6ab6e04f13a347a6bc0dc1099ee8d3f4/users/78ea77f42eab456e8e5ba45228f19106/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:50.433697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6dbab-d0d8-483a-8ac5-3f190a324a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ab6e04f13a347a6bc0dc1099ee8d3f4, user_id=78ea77f42eab456e8e5ba45228f19106, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:50.436472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e6dbab-d0d8-483a-8ac5-3f190a324a19 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:50.448502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 916/1804] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:50 2022] PUT /identity/v3/projects/6ab6e04f13a347a6bc0dc1099ee8d3f4/users/78ea77f42eab456e8e5ba45228f19106/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:50.460467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df30e8-2c1e-45bb-8f38-daa79581f54e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:50.460765 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df30e8-2c1e-45bb-8f38-daa79581f54e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:50.461195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df30e8-2c1e-45bb-8f38-daa79581f54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:50.496085 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-45df30e8-2c1e-45bb-8f38-daa79581f54e None None] MFA Rules not processed for user `78ea77f42eab456e8e5ba45228f19106`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:50.497922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-45df30e8-2c1e-45bb-8f38-daa79581f54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:50.604390 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 889/1805] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:50.990733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6dbab-d0d8-483a-8ac5-3f190a324a19 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:50.992711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d2e6dbab-d0d8-483a-8ac5-3f190a324a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:51.105341 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:51.148894 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e6dbab-d0d8-483a-8ac5-3f190a324a19 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:51.153668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42efbf8-6198-413a-8af1-b96b68971de2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:51.154414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efbf8-6198-413a-8af1-b96b68971de2 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:51.155106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efbf8-6198-413a-8af1-b96b68971de2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:51.155396 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42efbf8-6198-413a-8af1-b96b68971de2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:51.170586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42efbf8-6198-413a-8af1-b96b68971de2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:51.184383 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42efbf8-6198-413a-8af1-b96b68971de2 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:51.210627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 917/1806] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:51.259581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df30e8-2c1e-45bb-8f38-daa79581f54e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:51.396643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df30e8-2c1e-45bb-8f38-daa79581f54e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:51.396643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:51.396643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:51.396643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:51.397030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] PATH_INFO: `/v3/users/047dea136c4b48dd999c4998b8af4963` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:51.397306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=047dea136c4b48dd999c4998b8af4963)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:51.399350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:51.591475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '047dea136c4b48dd999c4998b8af4963'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:51.688858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '047dea136c4b48dd999c4998b8af4963'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:51.689489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '047dea136c4b48dd999c4998b8af4963'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:51.690134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '047dea136c4b48dd999c4998b8af4963'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:51.698004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '047dea136c4b48dd999c4998b8af4963'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:51.722021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 890/1807] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:51 2022] DELETE /identity/v3/users/047dea136c4b48dd999c4998b8af4963 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:51.748919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4532e-700e-4160-acfa-0f75da832185 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:51.748919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4532e-700e-4160-acfa-0f75da832185 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:51.748919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4532e-700e-4160-acfa-0f75da832185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:51.748919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-f7d4532e-700e-4160-acfa-0f75da832185 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:51.749840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f7d4532e-700e-4160-acfa-0f75da832185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:51.821814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 918/1808] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:51.832000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:51.834400 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:51.914422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdd52c-1685-405a-b2b6-4d959fb2efa3 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:51.914422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6131747-b5ee-43e1-bc3e-b3b8c2036c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:51.914422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6131747-b5ee-43e1-bc3e-b3b8c2036c18 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:51.914422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6131747-b5ee-43e1-bc3e-b3b8c2036c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:51.915145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6131747-b5ee-43e1-bc3e-b3b8c2036c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:51.915145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6131747-b5ee-43e1-bc3e-b3b8c2036c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:51.915145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6131747-b5ee-43e1-bc3e-b3b8c2036c18 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:51.927938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 891/1809] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:52.262562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:52.659663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4532e-700e-4160-acfa-0f75da832185 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:52.659663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f7d4532e-700e-4160-acfa-0f75da832185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:52.682209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4532e-700e-4160-acfa-0f75da832185 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:52.687669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:52.688609 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:52.688877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:52.689248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] PATH_INFO: `/v3/projects/0e6ee18d803549898d6905a26c67f422` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:52.690509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e6ee18d803549898d6905a26c67f422)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:52.692583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:52.839488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e6ee18d803549898d6905a26c67f422', 'request_id': 'req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:52.844801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e6ee18d803549898d6905a26c67f422', 'request_id': 'req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:52.963181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e6ee18d803549898d6905a26c67f422', 'request_id': 'req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:52.975598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] The token cache is being invalidate because project 0e6ee18d803549898d6905a26c67f422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:52.975932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:52.976567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:52.978041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 919/1810] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:52 2022] DELETE /identity/v3/projects/0e6ee18d803549898d6905a26c67f422 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:52.982167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6131747-b5ee-43e1-bc3e-b3b8c2036c18 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:52.991646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6131747-b5ee-43e1-bc3e-b3b8c2036c18 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:53.051060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:53.051060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:53.051060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:53.051060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] PATH_INFO: `/v3/users/78ea77f42eab456e8e5ba45228f19106` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:53.051956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ea77f42eab456e8e5ba45228f19106)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:53.053866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:53.308294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ea77f42eab456e8e5ba45228f19106'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:53.374089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ea77f42eab456e8e5ba45228f19106'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:53.374826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ea77f42eab456e8e5ba45228f19106'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:53.375460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ea77f42eab456e8e5ba45228f19106'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:53.384592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ea77f42eab456e8e5ba45228f19106'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:53.416986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 892/1811] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:52 2022] DELETE /identity/v3/users/78ea77f42eab456e8e5ba45228f19106 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:53.596475 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:53.598117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:53.669845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e50ee-7923-44e0-981d-4a86bba4bdb7 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:53.733820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7f355-fc75-4958-acd2-efe6e55ce01c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:53.734477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7f355-fc75-4958-acd2-efe6e55ce01c service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:53.734809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7f355-fc75-4958-acd2-efe6e55ce01c service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:53.735089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7f355-fc75-4958-acd2-efe6e55ce01c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:53.737027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-36b7f355-fc75-4958-acd2-efe6e55ce01c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:53.818030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7f355-fc75-4958-acd2-efe6e55ce01c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:53.819684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7f355-fc75-4958-acd2-efe6e55ce01c service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:53.833495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 920/1812] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:53.880883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:53.883555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:53.987477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49490c-767e-4e55-90ac-d17b5ad77e07 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:53.992853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:53.993532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:53.993826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:53.994102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] PATH_INFO: `/v3/projects/6ab6e04f13a347a6bc0dc1099ee8d3f4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:53.995252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ab6e04f13a347a6bc0dc1099ee8d3f4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:53.997197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:54.142157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ab6e04f13a347a6bc0dc1099ee8d3f4', 'request_id': 'req-bee7371e-fdf6-49cb-8d91-a154b9b444ab'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:54.172103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:54.222879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ab6e04f13a347a6bc0dc1099ee8d3f4', 'request_id': 'req-bee7371e-fdf6-49cb-8d91-a154b9b444ab'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:54.230058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ab6e04f13a347a6bc0dc1099ee8d3f4', 'request_id': 'req-bee7371e-fdf6-49cb-8d91-a154b9b444ab'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:54.241030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] The token cache is being invalidate because project 6ab6e04f13a347a6bc0dc1099ee8d3f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:54.241616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:54.242254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:54.243846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 893/1813] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:53 2022] DELETE /identity/v3/projects/6ab6e04f13a347a6bc0dc1099ee8d3f4 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:54.249402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268ff37-3225-405c-bc76-2dff85472aee None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:54.249695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268ff37-3225-405c-bc76-2dff85472aee None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:54.249973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268ff37-3225-405c-bc76-2dff85472aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:54.272064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-c268ff37-3225-405c-bc76-2dff85472aee None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:54.273304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c268ff37-3225-405c-bc76-2dff85472aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:54.337884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:54.339357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:54.349319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 921/1814] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:54.357029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1eab-c72b-401c-bf7a-ab9a6ef3dd2b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:54.357369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1eab-c72b-401c-bf7a-ab9a6ef3dd2b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:54.357697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd1eab-c72b-401c-bf7a-ab9a6ef3dd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:54.380886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-45cd1eab-c72b-401c-bf7a-ab9a6ef3dd2b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:54.382016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-45cd1eab-c72b-401c-bf7a-ab9a6ef3dd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:54.390202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 922/1815] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:54.394853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd1eab-c72b-401c-bf7a-ab9a6ef3dd2b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:54.403975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd1eab-c72b-401c-bf7a-ab9a6ef3dd2b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:54.408741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b079e97-3b5b-48d1-8e85-5e632ae4d9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:54.409488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b079e97-3b5b-48d1-8e85-5e632ae4d9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:54.409608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b079e97-3b5b-48d1-8e85-5e632ae4d9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:54.410535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b079e97-3b5b-48d1-8e85-5e632ae4d9c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:54.411136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b079e97-3b5b-48d1-8e85-5e632ae4d9c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:54.411439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b079e97-3b5b-48d1-8e85-5e632ae4d9c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:54.413080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b079e97-3b5b-48d1-8e85-5e632ae4d9c5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:54.431632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 923/1816] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:22:54.432555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b079e97-3b5b-48d1-8e85-5e632ae4d9c5 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:54.439440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7371e-fdf6-49cb-8d91-a154b9b444ab admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:54.442668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b079e97-3b5b-48d1-8e85-5e632ae4d9c5 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:54.444359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a0cbcc-4239-4b3f-8ccb-4f44d1d21a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:54.445045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0cbcc-4239-4b3f-8ccb-4f44d1d21a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:54.445335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0cbcc-4239-4b3f-8ccb-4f44d1d21a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:54.445619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a0cbcc-4239-4b3f-8ccb-4f44d1d21a19 admin admin] PATH_INFO: `/v3/users/d2f6dafa676144cfb3e501c72860e50c` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:54.446826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0cbcc-4239-4b3f-8ccb-4f44d1d21a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2f6dafa676144cfb3e501c72860e50c)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:54.447574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0318b-074b-4996-bcfa-46af84aabf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:54.448205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0318b-074b-4996-bcfa-46af84aabf34 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:54.448483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0318b-074b-4996-bcfa-46af84aabf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:54.448753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0318b-074b-4996-bcfa-46af84aabf34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:54.449638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a0cbcc-4239-4b3f-8ccb-4f44d1d21a19 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:54.449887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0318b-074b-4996-bcfa-46af84aabf34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:54.451244 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0318b-074b-4996-bcfa-46af84aabf34 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:54.594853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 924/1817] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:54 2022] POST /identity/v3/projects => generated 409 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:54.598811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0318b-074b-4996-bcfa-46af84aabf34 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:54.688085 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-33a0cbcc-4239-4b3f-8ccb-4f44d1d21a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2f6dafa676144cfb3e501c72860e50c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:54.701325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0318b-074b-4996-bcfa-46af84aabf34 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:54.705993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e301711-c3fb-43a8-9f08-5025429daa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:54.707730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301711-c3fb-43a8-9f08-5025429daa48 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:54.712734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301711-c3fb-43a8-9f08-5025429daa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:54.712734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e301711-c3fb-43a8-9f08-5025429daa48 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:54.712734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e301711-c3fb-43a8-9f08-5025429daa48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:54.712734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e301711-c3fb-43a8-9f08-5025429daa48 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:54.759399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-33a0cbcc-4239-4b3f-8ccb-4f44d1d21a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2f6dafa676144cfb3e501c72860e50c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:54.760074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-33a0cbcc-4239-4b3f-8ccb-4f44d1d21a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2f6dafa676144cfb3e501c72860e50c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:54.760702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-33a0cbcc-4239-4b3f-8ccb-4f44d1d21a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2f6dafa676144cfb3e501c72860e50c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:54.775229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-33a0cbcc-4239-4b3f-8ccb-4f44d1d21a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2f6dafa676144cfb3e501c72860e50c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:54.788002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 894/1818] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:54 2022] DELETE /identity/v3/users/d2f6dafa676144cfb3e501c72860e50c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:22:54.792434 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27e069-f2a0-4b0d-9422-6b1052c7f760 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:54.793171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27e069-f2a0-4b0d-9422-6b1052c7f760 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:54.793419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a27e069-f2a0-4b0d-9422-6b1052c7f760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:54.817878 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-3a27e069-f2a0-4b0d-9422-6b1052c7f760 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:54.819219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3a27e069-f2a0-4b0d-9422-6b1052c7f760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:54.828019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 925/1819] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:54 2022] POST /identity/v3/users => generated 305 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:54.832653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e301711-c3fb-43a8-9f08-5025429daa48 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:54.835845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-3e301711-c3fb-43a8-9f08-5025429daa48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:54.856580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e301711-c3fb-43a8-9f08-5025429daa48 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:54.901040 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 895/1820] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:54.910932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27e069-f2a0-4b0d-9422-6b1052c7f760 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:54.913420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3a27e069-f2a0-4b0d-9422-6b1052c7f760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:54.914483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:54.962414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d285715-67a0-479a-92e1-2e224bad305b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:54.963338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d285715-67a0-479a-92e1-2e224bad305b admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:54.963626 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d285715-67a0-479a-92e1-2e224bad305b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:54.963994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d285715-67a0-479a-92e1-2e224bad305b admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:54.964654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d285715-67a0-479a-92e1-2e224bad305b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:54.966805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d285715-67a0-479a-92e1-2e224bad305b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:54.977798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 926/1821] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:22:54.981472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d285715-67a0-479a-92e1-2e224bad305b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:54.995398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d285715-67a0-479a-92e1-2e224bad305b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:55.020638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71caf66-20df-43eb-96a7-6d7e1e507e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:55.021261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71caf66-20df-43eb-96a7-6d7e1e507e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:55.021537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71caf66-20df-43eb-96a7-6d7e1e507e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:55.021806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71caf66-20df-43eb-96a7-6d7e1e507e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:55.022703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71caf66-20df-43eb-96a7-6d7e1e507e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:55.025906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71caf66-20df-43eb-96a7-6d7e1e507e4c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:55.041110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a27e069-f2a0-4b0d-9422-6b1052c7f760 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:55.044306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 927/1822] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:22:55.045550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71caf66-20df-43eb-96a7-6d7e1e507e4c admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:55.049449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e68579-f1a3-4231-8f34-26605a3ff3ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:55.049449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e68579-f1a3-4231-8f34-26605a3ff3ac service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:55.049449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e68579-f1a3-4231-8f34-26605a3ff3ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:55.049449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e68579-f1a3-4231-8f34-26605a3ff3ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:55.056657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71caf66-20df-43eb-96a7-6d7e1e507e4c admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:55.060806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e68579-f1a3-4231-8f34-26605a3ff3ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:55.061743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a249b-c2f1-4de4-bbd2-b8ff8d537604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:55.062254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e68579-f1a3-4231-8f34-26605a3ff3ac service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:55.062405 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a249b-c2f1-4de4-bbd2-b8ff8d537604 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:55.062675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a249b-c2f1-4de4-bbd2-b8ff8d537604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:55.062953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a249b-c2f1-4de4-bbd2-b8ff8d537604 admin admin] PATH_INFO: `/v3/projects/a4ba5a53fa8e480e88d4e953a11ceae0/users/80c9b81ca37549729dee71c163a8819c/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:55.073918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 896/1823] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:22:55.091158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a249b-c2f1-4de4-bbd2-b8ff8d537604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ba5a53fa8e480e88d4e953a11ceae0, user_id=80c9b81ca37549729dee71c163a8819c, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:55.108177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a249b-c2f1-4de4-bbd2-b8ff8d537604 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:55.188833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 928/1824] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:55 2022] PUT /identity/v3/projects/a4ba5a53fa8e480e88d4e953a11ceae0/users/80c9b81ca37549729dee71c163a8819c/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:55.203612 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037af03c-2fa4-4718-8fa8-a21376b827c6 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:55.203932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037af03c-2fa4-4718-8fa8-a21376b827c6 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:55.204219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037af03c-2fa4-4718-8fa8-a21376b827c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:55.239483 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-037af03c-2fa4-4718-8fa8-a21376b827c6 None None] MFA Rules not processed for user `80c9b81ca37549729dee71c163a8819c`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:55.240742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-037af03c-2fa4-4718-8fa8-a21376b827c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:55.308033 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 897/1825] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:55 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:55.312617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a249b-c2f1-4de4-bbd2-b8ff8d537604 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:55.406851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a249b-c2f1-4de4-bbd2-b8ff8d537604 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:55.411708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d050ead-f7ac-43cb-92ef-5b05f26ec906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:55.412415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d050ead-f7ac-43cb-92ef-5b05f26ec906 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:55.412632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d050ead-f7ac-43cb-92ef-5b05f26ec906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:55.412953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d050ead-f7ac-43cb-92ef-5b05f26ec906 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:55.413817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d050ead-f7ac-43cb-92ef-5b05f26ec906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:55.415924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d050ead-f7ac-43cb-92ef-5b05f26ec906 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:55.507212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 929/1826] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:55 2022] POST /identity/v3/projects => generated 405 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:55.511320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037af03c-2fa4-4718-8fa8-a21376b827c6 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:55.521998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037af03c-2fa4-4718-8fa8-a21376b827c6 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:55.578156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f313a7-5d86-405c-9b83-f217978345fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:55.578952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f313a7-5d86-405c-9b83-f217978345fe admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:55.579236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f313a7-5d86-405c-9b83-f217978345fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:55.579503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f313a7-5d86-405c-9b83-f217978345fe admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:55.580518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f313a7-5d86-405c-9b83-f217978345fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:55.584058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f313a7-5d86-405c-9b83-f217978345fe admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:55.588529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d050ead-f7ac-43cb-92ef-5b05f26ec906 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:55.588529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1d050ead-f7ac-43cb-92ef-5b05f26ec906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:55.600296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d050ead-f7ac-43cb-92ef-5b05f26ec906 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:55.611590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:55.611590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:55.611590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:55.611590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] PATH_INFO: `/v3/projects/af4a76490ae847c3a6d679f68f1d6582` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:55.612308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=af4a76490ae847c3a6d679f68f1d6582)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:55.612308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:55.688410 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 898/1827] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:55 2022] POST /identity/v3/users => generated 309 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:55.693275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f313a7-5d86-405c-9b83-f217978345fe admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:55.705521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f313a7-5d86-405c-9b83-f217978345fe admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:55.736571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af4a76490ae847c3a6d679f68f1d6582', 'request_id': 'req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:55.744263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af4a76490ae847c3a6d679f68f1d6582', 'request_id': 'req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:55.774263 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebb6c4-33d8-4103-bd45-26faec0fc5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:55.774892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebb6c4-33d8-4103-bd45-26faec0fc5fb admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:55.775176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebb6c4-33d8-4103-bd45-26faec0fc5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:55.775452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebb6c4-33d8-4103-bd45-26faec0fc5fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:55.776138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ebb6c4-33d8-4103-bd45-26faec0fc5fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:55.777884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ebb6c4-33d8-4103-bd45-26faec0fc5fb admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:55.788681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 899/1828] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:22:55.792623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebb6c4-33d8-4103-bd45-26faec0fc5fb admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:55.802650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ebb6c4-33d8-4103-bd45-26faec0fc5fb admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:55.807366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d65f1-a0a7-4302-92c5-1e09842adc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:55.808051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d65f1-a0a7-4302-92c5-1e09842adc11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:55.808337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d65f1-a0a7-4302-92c5-1e09842adc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:55.808618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d65f1-a0a7-4302-92c5-1e09842adc11 admin admin] PATH_INFO: `/v3/projects/657d417959354d3fb1a409a77d58c414/users/e16c844eaea5479aa1c16c1debbd8016/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:55.814339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af4a76490ae847c3a6d679f68f1d6582', 'request_id': 'req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:55.836083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d65f1-a0a7-4302-92c5-1e09842adc11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=657d417959354d3fb1a409a77d58c414, user_id=e16c844eaea5479aa1c16c1debbd8016, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:55.836083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d65f1-a0a7-4302-92c5-1e09842adc11 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:55.838063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] The token cache is being invalidate because project af4a76490ae847c3a6d679f68f1d6582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:55.838358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:55.839028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:55.840554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 930/1829] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:55 2022] DELETE /identity/v3/projects/af4a76490ae847c3a6d679f68f1d6582 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:55.844435 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:55.845733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:55.858273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16bbad-2ac6-47cf-b5b6-8c765a6d3bcd admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:55.863326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:55.864033 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:55.864334 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:55.864594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] PATH_INFO: `/v3/users/83432b147a30487cbcb84b2940a6df41` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:55.865717 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=83432b147a30487cbcb84b2940a6df41)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:55.867614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:56.022776 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 900/1830] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:55 2022] PUT /identity/v3/projects/657d417959354d3fb1a409a77d58c414/users/e16c844eaea5479aa1c16c1debbd8016/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:22:56.026911 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d65f1-a0a7-4302-92c5-1e09842adc11 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:56.028916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-007d65f1-a0a7-4302-92c5-1e09842adc11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:56.095614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d65f1-a0a7-4302-92c5-1e09842adc11 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:56.100457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c16896-c44a-4dfa-b8de-2a0bb0951dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:56.101056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c16896-c44a-4dfa-b8de-2a0bb0951dda admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:56.101337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c16896-c44a-4dfa-b8de-2a0bb0951dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:56.101612 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c16896-c44a-4dfa-b8de-2a0bb0951dda admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:56.102257 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c16896-c44a-4dfa-b8de-2a0bb0951dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:56.103919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c16896-c44a-4dfa-b8de-2a0bb0951dda admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:56.111569 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:56.115188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 901/1831] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:22:56.119165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c16896-c44a-4dfa-b8de-2a0bb0951dda admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:56.130514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c16896-c44a-4dfa-b8de-2a0bb0951dda admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:56.135299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439b465-aa8c-4f9e-b8e6-45f4585a8511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:56.135978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439b465-aa8c-4f9e-b8e6-45f4585a8511 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:56.136267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439b465-aa8c-4f9e-b8e6-45f4585a8511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:56.136542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439b465-aa8c-4f9e-b8e6-45f4585a8511 admin admin] PATH_INFO: `/v3/projects/657d417959354d3fb1a409a77d58c414/users/e16c844eaea5479aa1c16c1debbd8016/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:56.138502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439b465-aa8c-4f9e-b8e6-45f4585a8511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=657d417959354d3fb1a409a77d58c414, user_id=e16c844eaea5479aa1c16c1debbd8016, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:56.141186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4439b465-aa8c-4f9e-b8e6-45f4585a8511 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:56.153063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 902/1832] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:56 2022] PUT /identity/v3/projects/657d417959354d3fb1a409a77d58c414/users/e16c844eaea5479aa1c16c1debbd8016/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:22:56.154586 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:56.162543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83432b147a30487cbcb84b2940a6df41'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:56.166193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4139e-4ea6-458b-9f02-f86ef7074881 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:56.166914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4139e-4ea6-458b-9f02-f86ef7074881 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:56.167201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4139e-4ea6-458b-9f02-f86ef7074881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:56.198313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-31d4139e-4ea6-458b-9f02-f86ef7074881 None None] MFA Rules not processed for user `e16c844eaea5479aa1c16c1debbd8016`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:56.199564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-31d4139e-4ea6-458b-9f02-f86ef7074881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:56.219882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83432b147a30487cbcb84b2940a6df41'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:56.219882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83432b147a30487cbcb84b2940a6df41'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:56.219882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83432b147a30487cbcb84b2940a6df41'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:56.226152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83432b147a30487cbcb84b2940a6df41'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:56.251942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 931/1833] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:22:55 2022] DELETE /identity/v3/users/83432b147a30487cbcb84b2940a6df41 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:22:56.277509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 903/1834] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:56 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:56.288896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:56.290328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:56.364145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbcc46-10ff-4b5d-826b-639ba1dde8aa admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:56.367707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9da24c-9b3f-46e4-81c8-97312d69841e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:56.368546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da24c-9b3f-46e4-81c8-97312d69841e service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:56.368829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da24c-9b3f-46e4-81c8-97312d69841e service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:56.369107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9da24c-9b3f-46e4-81c8-97312d69841e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:56.429126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9da24c-9b3f-46e4-81c8-97312d69841e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:56.430557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9da24c-9b3f-46e4-81c8-97312d69841e service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:56.439881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 932/1835] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:56 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:56.727955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4139e-4ea6-458b-9f02-f86ef7074881 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:56.729701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-31d4139e-4ea6-458b-9f02-f86ef7074881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:56.800592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4139e-4ea6-458b-9f02-f86ef7074881 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:56.883038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:56.884363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:56.884804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:56.885226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] PATH_INFO: `/v3/projects/e9021af8696540108d6decc797c966fa` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:56.886764 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9021af8696540108d6decc797c966fa)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:56.889847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:56.938557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9da24c-9b3f-46e4-81c8-97312d69841e service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:56.952146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9da24c-9b3f-46e4-81c8-97312d69841e service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:56.955961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71628c-9d79-4d54-8edb-257cd9876a46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:56.957119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71628c-9d79-4d54-8edb-257cd9876a46 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:56.957842 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71628c-9d79-4d54-8edb-257cd9876a46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:56.958304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71628c-9d79-4d54-8edb-257cd9876a46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:56.960786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-3d71628c-9d79-4d54-8edb-257cd9876a46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:57.049248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9021af8696540108d6decc797c966fa', 'request_id': 'req-3a2cde58-842b-4e13-9d53-1e75f003dd10'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:57.049876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71628c-9d79-4d54-8edb-257cd9876a46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:57.051673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71628c-9d79-4d54-8edb-257cd9876a46 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:57.062344 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 933/1836] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:56 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:57.337029 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:57.913856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9021af8696540108d6decc797c966fa', 'request_id': 'req-3a2cde58-842b-4e13-9d53-1e75f003dd10'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:57.920674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9021af8696540108d6decc797c966fa', 'request_id': 'req-3a2cde58-842b-4e13-9d53-1e75f003dd10'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:57.934573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] The token cache is being invalidate because project e9021af8696540108d6decc797c966fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:22:57.934975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:57.935615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3a2cde58-842b-4e13-9d53-1e75f003dd10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:22:57.938327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 904/1837] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:22:56 2022] DELETE /identity/v3/projects/e9021af8696540108d6decc797c966fa => generated 0 bytes in 1211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:57.943189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c2d1e-dc63-4076-a309-c1c4c6cb94b2 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:57.943701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c2d1e-dc63-4076-a309-c1c4c6cb94b2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:57.943829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c2d1e-dc63-4076-a309-c1c4c6cb94b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:57.964216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e38c2d1e-dc63-4076-a309-c1c4c6cb94b2 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:57.965551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e38c2d1e-dc63-4076-a309-c1c4c6cb94b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:58.026624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 934/1838] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:58.036238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04316351-3726-45aa-97b6-ea0027caedff None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:58.036524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04316351-3726-45aa-97b6-ea0027caedff None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:58.036798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04316351-3726-45aa-97b6-ea0027caedff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:58.060343 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-04316351-3726-45aa-97b6-ea0027caedff None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:58.061566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-04316351-3726-45aa-97b6-ea0027caedff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:58.098675 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 905/1839] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:58.103614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c2d1e-dc63-4076-a309-c1c4c6cb94b2 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:58.126017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c2d1e-dc63-4076-a309-c1c4c6cb94b2 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:58.139428 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40606673-0358-46b2-a135-4ce3a71f6629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:58.140333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40606673-0358-46b2-a135-4ce3a71f6629 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:58.140805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40606673-0358-46b2-a135-4ce3a71f6629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:58.141333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40606673-0358-46b2-a135-4ce3a71f6629 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:58.142290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40606673-0358-46b2-a135-4ce3a71f6629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:22:58.142602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40606673-0358-46b2-a135-4ce3a71f6629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:58.144795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40606673-0358-46b2-a135-4ce3a71f6629 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:58.171974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 935/1840] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:22:58.174172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04316351-3726-45aa-97b6-ea0027caedff None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:58.190249 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04316351-3726-45aa-97b6-ea0027caedff None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:58.203707 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77959908-d5ad-4cb7-ac3f-5f8fa9dcbece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:58.204544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77959908-d5ad-4cb7-ac3f-5f8fa9dcbece admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:58.204906 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77959908-d5ad-4cb7-ac3f-5f8fa9dcbece admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:58.205260 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77959908-d5ad-4cb7-ac3f-5f8fa9dcbece admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:58.206934 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77959908-d5ad-4cb7-ac3f-5f8fa9dcbece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:58.208994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77959908-d5ad-4cb7-ac3f-5f8fa9dcbece admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:58.326216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 906/1841] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:58 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:58.326392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:58.327586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40606673-0358-46b2-a135-4ce3a71f6629 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:58.338741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40606673-0358-46b2-a135-4ce3a71f6629 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:58.403125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913de11f-30fe-4de6-84c2-c0b20932ab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:58.404297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913de11f-30fe-4de6-84c2-c0b20932ab6f admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:58.404652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913de11f-30fe-4de6-84c2-c0b20932ab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:58.408416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913de11f-30fe-4de6-84c2-c0b20932ab6f admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:58.408416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913de11f-30fe-4de6-84c2-c0b20932ab6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:58.408416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913de11f-30fe-4de6-84c2-c0b20932ab6f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:58.537128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 936/1842] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:58 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:58.547644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77959908-d5ad-4cb7-ac3f-5f8fa9dcbece admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:58.559352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77959908-d5ad-4cb7-ac3f-5f8fa9dcbece admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:58.564478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f1c4a-628d-4d11-a811-e9dcc53f82d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:58.565214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1c4a-628d-4d11-a811-e9dcc53f82d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:58.565494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1c4a-628d-4d11-a811-e9dcc53f82d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:58.566284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f1c4a-628d-4d11-a811-e9dcc53f82d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:58.567137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f1c4a-628d-4d11-a811-e9dcc53f82d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:58.568916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f1c4a-628d-4d11-a811-e9dcc53f82d3 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:58.583436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 907/1843] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:58.586377 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913de11f-30fe-4de6-84c2-c0b20932ab6f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:58.602379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913de11f-30fe-4de6-84c2-c0b20932ab6f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:58.607167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e2c4b-201c-47e3-8852-b2fe5b542982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:58.607806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e2c4b-201c-47e3-8852-b2fe5b542982 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:58.608244 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e2c4b-201c-47e3-8852-b2fe5b542982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:58.608527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e2c4b-201c-47e3-8852-b2fe5b542982 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:58.609200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e2c4b-201c-47e3-8852-b2fe5b542982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:58.610798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e2c4b-201c-47e3-8852-b2fe5b542982 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:58.636065 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 937/1844] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:58.637270 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f1c4a-628d-4d11-a811-e9dcc53f82d3 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:58.649044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f1c4a-628d-4d11-a811-e9dcc53f82d3 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:58.658634 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c55ca2-f1fa-49f4-b769-dcc60d7748d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:58.658634 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c55ca2-f1fa-49f4-b769-dcc60d7748d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:58.658634 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c55ca2-f1fa-49f4-b769-dcc60d7748d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:58.658634 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c55ca2-f1fa-49f4-b769-dcc60d7748d7 admin admin] PATH_INFO: `/v3/projects/005c5a91a8e54f778ed19ff1231f7639/users/bf60a530c54946a7b4d62392a1a4ba4a/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:58.688683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c55ca2-f1fa-49f4-b769-dcc60d7748d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=005c5a91a8e54f778ed19ff1231f7639, user_id=bf60a530c54946a7b4d62392a1a4ba4a, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:58.693774 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c55ca2-f1fa-49f4-b769-dcc60d7748d7 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:58.811620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 908/1845] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:58 2022] PUT /identity/v3/projects/005c5a91a8e54f778ed19ff1231f7639/users/bf60a530c54946a7b4d62392a1a4ba4a/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:58.822732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1550-924e-46ca-beeb-6ce67cb25e00 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:58.823131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1550-924e-46ca-beeb-6ce67cb25e00 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:58.823415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1550-924e-46ca-beeb-6ce67cb25e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:58.853337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-5d6d1550-924e-46ca-beeb-6ce67cb25e00 None None] MFA Rules not processed for user `bf60a530c54946a7b4d62392a1a4ba4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:58.855358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5d6d1550-924e-46ca-beeb-6ce67cb25e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:58.921847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 938/1846] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:22:58.927156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c55ca2-f1fa-49f4-b769-dcc60d7748d7 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:58.992432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c55ca2-f1fa-49f4-b769-dcc60d7748d7 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:59.002217 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea01aa-15c6-472e-b8f9-9daa7f64b63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:59.002893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea01aa-15c6-472e-b8f9-9daa7f64b63f admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:59.003171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea01aa-15c6-472e-b8f9-9daa7f64b63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:59.004047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ea01aa-15c6-472e-b8f9-9daa7f64b63f admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:59.004740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea01aa-15c6-472e-b8f9-9daa7f64b63f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:59.006935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ea01aa-15c6-472e-b8f9-9daa7f64b63f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:59.145272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 909/1847] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:22:58 2022] POST /identity/v3/projects => generated 375 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:59.146339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d1550-924e-46ca-beeb-6ce67cb25e00 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:59.157147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d1550-924e-46ca-beeb-6ce67cb25e00 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:59.214447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4f16d-b884-4959-b412-01914293dcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:59.215061 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4f16d-b884-4959-b412-01914293dcee admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:59.215336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4f16d-b884-4959-b412-01914293dcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:59.215612 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4f16d-b884-4959-b412-01914293dcee admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:59.216363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4f16d-b884-4959-b412-01914293dcee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:59.218396 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4f16d-b884-4959-b412-01914293dcee admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:59.285469 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 939/1848] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:22:59 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:22:59.289427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea01aa-15c6-472e-b8f9-9daa7f64b63f admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:59.300079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ea01aa-15c6-472e-b8f9-9daa7f64b63f admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:59.305456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845098af-2a8d-453c-a632-9e85294b669e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:59.306039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845098af-2a8d-453c-a632-9e85294b669e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:59.306471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845098af-2a8d-453c-a632-9e85294b669e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:59.306809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845098af-2a8d-453c-a632-9e85294b669e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:59.307403 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845098af-2a8d-453c-a632-9e85294b669e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:59.309015 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845098af-2a8d-453c-a632-9e85294b669e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:59.319758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 910/1849] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:59.324016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4f16d-b884-4959-b412-01914293dcee admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:59.334768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4f16d-b884-4959-b412-01914293dcee admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:59.339675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e466-e105-4b34-81cd-54bcac748f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:59.340366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e466-e105-4b34-81cd-54bcac748f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:59.340651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e466-e105-4b34-81cd-54bcac748f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:59.341010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197e466-e105-4b34-81cd-54bcac748f59 admin admin] PATH_INFO: `/v3/projects/d939f1adaa5545cba7fb7ebbd950a26e/users/8959b2186e3a408c858059ccbfc02355/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:59.356936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197e466-e105-4b34-81cd-54bcac748f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d939f1adaa5545cba7fb7ebbd950a26e, user_id=8959b2186e3a408c858059ccbfc02355, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:59.359562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8197e466-e105-4b34-81cd-54bcac748f59 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:59.439757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 940/1850] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:59 2022] PUT /identity/v3/projects/d939f1adaa5545cba7fb7ebbd950a26e/users/8959b2186e3a408c858059ccbfc02355/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:59.443651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845098af-2a8d-453c-a632-9e85294b669e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:59.453487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845098af-2a8d-453c-a632-9e85294b669e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:59.510309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec956c6-53be-4c58-879f-b23df2d86557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:59.510914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec956c6-53be-4c58-879f-b23df2d86557 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:59.511198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec956c6-53be-4c58-879f-b23df2d86557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:59.511477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec956c6-53be-4c58-879f-b23df2d86557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:59.512159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec956c6-53be-4c58-879f-b23df2d86557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:59.513779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec956c6-53be-4c58-879f-b23df2d86557 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:59.523272 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 911/1851] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:22:59.527611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e466-e105-4b34-81cd-54bcac748f59 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:59.538461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8197e466-e105-4b34-81cd-54bcac748f59 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:59.543300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff9750-831a-4045-9adf-1118075a0725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:59.543985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff9750-831a-4045-9adf-1118075a0725 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:59.544266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff9750-831a-4045-9adf-1118075a0725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:59.544541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff9750-831a-4045-9adf-1118075a0725 admin admin] PATH_INFO: `/v3/projects/d939f1adaa5545cba7fb7ebbd950a26e/users/8959b2186e3a408c858059ccbfc02355/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:59.546468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff9750-831a-4045-9adf-1118075a0725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d939f1adaa5545cba7fb7ebbd950a26e, user_id=8959b2186e3a408c858059ccbfc02355, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:59.549482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff9750-831a-4045-9adf-1118075a0725 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:59.560150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 941/1852] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:22:59 2022] PUT /identity/v3/projects/d939f1adaa5545cba7fb7ebbd950a26e/users/8959b2186e3a408c858059ccbfc02355/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:22:59.569599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:22:59.572147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa68f8-bff6-4382-8b28-7d37420f9f46 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:59.572700 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa68f8-bff6-4382-8b28-7d37420f9f46 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:59.573009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa68f8-bff6-4382-8b28-7d37420f9f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:59.602306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-29aa68f8-bff6-4382-8b28-7d37420f9f46 None None] MFA Rules not processed for user `8959b2186e3a408c858059ccbfc02355`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:22:59.603519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-29aa68f8-bff6-4382-8b28-7d37420f9f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:59.673880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 912/1853] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:22:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:22:59.685180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff9750-831a-4045-9adf-1118075a0725 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:22:59.687015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-91ff9750-831a-4045-9adf-1118075a0725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:22:59.746113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff9750-831a-4045-9adf-1118075a0725 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:22:59.749757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f6349-317e-4061-8548-3c5dca8712ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:22:59.750365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f6349-317e-4061-8548-3c5dca8712ed service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:22:59.750642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f6349-317e-4061-8548-3c5dca8712ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:22:59.750915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f6349-317e-4061-8548-3c5dca8712ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:22:59.807085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f6349-317e-4061-8548-3c5dca8712ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:22:59.808538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f6349-317e-4061-8548-3c5dca8712ed service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:22:59.817775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 942/1854] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:22:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:22:59.862742 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:00.282917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:00.366104 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:00.545966 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad2706-5661-4683-9e48-f03f3ffd6f78 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:00.546638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad2706-5661-4683-9e48-f03f3ffd6f78 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:00.547609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad2706-5661-4683-9e48-f03f3ffd6f78 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:00.549340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 913/1855] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:23:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:23:00.559279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2a607-856b-46a2-933e-c167033adcfb None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:00.559472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2a607-856b-46a2-933e-c167033adcfb None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:00.559751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f2a607-856b-46a2-933e-c167033adcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:00.586458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b1f2a607-856b-46a2-933e-c167033adcfb None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:00.586458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b1f2a607-856b-46a2-933e-c167033adcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:00.701968 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 943/1856] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:23:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:00.718543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad2706-5661-4683-9e48-f03f3ffd6f78 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:00.720641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c9ad2706-5661-4683-9e48-f03f3ffd6f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:00.782636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad2706-5661-4683-9e48-f03f3ffd6f78 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:00.803943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb95365-6023-4824-b857-a21ba4bdd5ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:00.803943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb95365-6023-4824-b857-a21ba4bdd5ec service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:00.803943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb95365-6023-4824-b857-a21ba4bdd5ec service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:00.803943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb95365-6023-4824-b857-a21ba4bdd5ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:00.816975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb95365-6023-4824-b857-a21ba4bdd5ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:00.824834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb95365-6023-4824-b857-a21ba4bdd5ec service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:00.854288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 914/1857] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:23:01.093848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:01.459875 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:01.885155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:02.051419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:02.162602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:02.513545 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:02.576712 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:02.956879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:03.053937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:03.177239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:03.254419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:03.413879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:13.368343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=81 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:13.473305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=30 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:17.301388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2a607-856b-46a2-933e-c167033adcfb None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:17.302934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b1f2a607-856b-46a2-933e-c167033adcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:17.373240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f2a607-856b-46a2-933e-c167033adcfb None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:17.377802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:17.378437 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:17.378744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:17.379019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] PATH_INFO: `/v3/users/164749f76e654d519433c787e0f76868` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:17.380165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] RBAC: Authorizing `identity:delete_user(user_id=164749f76e654d519433c787e0f76868)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:17.382106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:17.576975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '164749f76e654d519433c787e0f76868'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:17.638162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '164749f76e654d519433c787e0f76868'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:17.638802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '164749f76e654d519433c787e0f76868'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:17.639433 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '164749f76e654d519433c787e0f76868'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:17.649574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '164749f76e654d519433c787e0f76868'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:17.670345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 944/1858] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:17 2022] DELETE /identity/v3/users/164749f76e654d519433c787e0f76868 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:17.674818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e421d1f-d1a9-4a91-ad0e-c003a8f9a3c6 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:17.675308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e421d1f-d1a9-4a91-ad0e-c003a8f9a3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:17.675498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e421d1f-d1a9-4a91-ad0e-c003a8f9a3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:17.695230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-2e421d1f-d1a9-4a91-ad0e-c003a8f9a3c6 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:17.696460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2e421d1f-d1a9-4a91-ad0e-c003a8f9a3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:17.777218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 915/1859] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:17.787422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:17.788874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:17.845926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff9899-d133-469c-b875-9050dea4ddad admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:17.850128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14085609-6d74-4c31-adfc-0577e52a8acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:17.850734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14085609-6d74-4c31-adfc-0577e52a8acd service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:17.851014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14085609-6d74-4c31-adfc-0577e52a8acd service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:17.851291 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14085609-6d74-4c31-adfc-0577e52a8acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:17.859961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14085609-6d74-4c31-adfc-0577e52a8acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:17.861384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14085609-6d74-4c31-adfc-0577e52a8acd service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:17.871220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 945/1860] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:23:18.300207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e421d1f-d1a9-4a91-ad0e-c003a8f9a3c6 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:18.302261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2e421d1f-d1a9-4a91-ad0e-c003a8f9a3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:18.320701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e421d1f-d1a9-4a91-ad0e-c003a8f9a3c6 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:18.320701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:18.320701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:18.320701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:18.321009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] PATH_INFO: `/v3/projects/8f94c57fb8d2487e9650710a859865c0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:18.321009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f94c57fb8d2487e9650710a859865c0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:18.323169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:18.436109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f94c57fb8d2487e9650710a859865c0', 'request_id': 'req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:18.509574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f94c57fb8d2487e9650710a859865c0', 'request_id': 'req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:18.516376 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f94c57fb8d2487e9650710a859865c0', 'request_id': 'req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:18.527601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] The token cache is being invalidate because project 8f94c57fb8d2487e9650710a859865c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:18.527931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:18.528691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:18.530209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 916/1861] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:18 2022] DELETE /identity/v3/projects/8f94c57fb8d2487e9650710a859865c0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:18.534127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14085609-6d74-4c31-adfc-0577e52a8acd service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:18.544763 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14085609-6d74-4c31-adfc-0577e52a8acd service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:18.564058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:18.604193 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5aac42-c009-4bc6-9d8a-980778d8bf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:18.604822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5aac42-c009-4bc6-9d8a-980778d8bf0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:18.605098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5aac42-c009-4bc6-9d8a-980778d8bf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:18.605484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5aac42-c009-4bc6-9d8a-980778d8bf0f admin admin] PATH_INFO: `/v3/users/1a9be282d08e45a882063b89ff0b8d5b` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:18.606588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5aac42-c009-4bc6-9d8a-980778d8bf0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a9be282d08e45a882063b89ff0b8d5b)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:18.608539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5aac42-c009-4bc6-9d8a-980778d8bf0f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:18.757547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ef5aac42-c009-4bc6-9d8a-980778d8bf0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a9be282d08e45a882063b89ff0b8d5b'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:18.824492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ef5aac42-c009-4bc6-9d8a-980778d8bf0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a9be282d08e45a882063b89ff0b8d5b'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:18.828110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ef5aac42-c009-4bc6-9d8a-980778d8bf0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a9be282d08e45a882063b89ff0b8d5b'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:18.828110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ef5aac42-c009-4bc6-9d8a-980778d8bf0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a9be282d08e45a882063b89ff0b8d5b'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:18.836906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ef5aac42-c009-4bc6-9d8a-980778d8bf0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a9be282d08e45a882063b89ff0b8d5b'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:18.854731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 946/1862] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:18 2022] DELETE /identity/v3/users/1a9be282d08e45a882063b89ff0b8d5b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:19.262802 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:19.264424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:19.279837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4276a8-b0a0-4b6a-990d-64c7ce16c6a1 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:19.351701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:19.352481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:19.352830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:19.353129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] PATH_INFO: `/v3/projects/a3cf9fdc42f042809d00252cc9e53a09` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:19.354263 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3cf9fdc42f042809d00252cc9e53a09)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:19.356231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:19.439828 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:19.456043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3cf9fdc42f042809d00252cc9e53a09', 'request_id': 'req-f005d81c-cff5-49ef-ae81-243d120697dd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:19.540819 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3cf9fdc42f042809d00252cc9e53a09', 'request_id': 'req-f005d81c-cff5-49ef-ae81-243d120697dd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:19.547919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3cf9fdc42f042809d00252cc9e53a09', 'request_id': 'req-f005d81c-cff5-49ef-ae81-243d120697dd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:19.558830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] The token cache is being invalidate because project a3cf9fdc42f042809d00252cc9e53a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:19.559138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:19.559805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f005d81c-cff5-49ef-ae81-243d120697dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:19.561309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 917/1863] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:19 2022] DELETE /identity/v3/projects/a3cf9fdc42f042809d00252cc9e53a09 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:19.567652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c83a6a-2b24-4af0-ae8d-3a6acb8782f1 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:19.568132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c83a6a-2b24-4af0-ae8d-3a6acb8782f1 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:19.568219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c83a6a-2b24-4af0-ae8d-3a6acb8782f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:19.588058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-a6c83a6a-2b24-4af0-ae8d-3a6acb8782f1 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:19.589292 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a6c83a6a-2b24-4af0-ae8d-3a6acb8782f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:19.650898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 947/1864] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:19.660596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa62a06-fb9f-47f2-a4be-26bdaeae1f22 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:19.660887 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa62a06-fb9f-47f2-a4be-26bdaeae1f22 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:19.661168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa62a06-fb9f-47f2-a4be-26bdaeae1f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:19.682836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-4aa62a06-fb9f-47f2-a4be-26bdaeae1f22 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:19.684120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4aa62a06-fb9f-47f2-a4be-26bdaeae1f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:19.692613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 918/1865] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:19.696879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c83a6a-2b24-4af0-ae8d-3a6acb8782f1 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:19.707321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c83a6a-2b24-4af0-ae8d-3a6acb8782f1 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:19.712022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272637b-8ea8-415c-8333-514ba387d54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:19.712654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272637b-8ea8-415c-8333-514ba387d54c admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:19.712937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272637b-8ea8-415c-8333-514ba387d54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:19.713217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272637b-8ea8-415c-8333-514ba387d54c admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:19.715938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272637b-8ea8-415c-8333-514ba387d54c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:23:19.715938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272637b-8ea8-415c-8333-514ba387d54c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:19.715938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272637b-8ea8-415c-8333-514ba387d54c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:19.727450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 948/1866] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:23:19.731756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa62a06-fb9f-47f2-a4be-26bdaeae1f22 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:19.741065 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa62a06-fb9f-47f2-a4be-26bdaeae1f22 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:19.745966 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336738d-3844-4dd9-977e-16e252bdb604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:19.746564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336738d-3844-4dd9-977e-16e252bdb604 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:19.746838 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336738d-3844-4dd9-977e-16e252bdb604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:19.747106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336738d-3844-4dd9-977e-16e252bdb604 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:19.747741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336738d-3844-4dd9-977e-16e252bdb604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:19.749731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336738d-3844-4dd9-977e-16e252bdb604 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:19.905620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 919/1867] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:19 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:19.909817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272637b-8ea8-415c-8333-514ba387d54c admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:19.919854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272637b-8ea8-415c-8333-514ba387d54c admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:19.946505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:19.977311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa3d2b-ad9f-41e8-b45a-9435d4911e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:19.978298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa3d2b-ad9f-41e8-b45a-9435d4911e78 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:19.978771 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa3d2b-ad9f-41e8-b45a-9435d4911e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:19.979373 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaa3d2b-ad9f-41e8-b45a-9435d4911e78 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:19.980248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa3d2b-ad9f-41e8-b45a-9435d4911e78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:19.982746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaa3d2b-ad9f-41e8-b45a-9435d4911e78 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:20.089831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 949/1868] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:19 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:20.094302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336738d-3844-4dd9-977e-16e252bdb604 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:20.106557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336738d-3844-4dd9-977e-16e252bdb604 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:20.115424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f616b-9194-49ea-94d4-cf16de55ab98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:20.116075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f616b-9194-49ea-94d4-cf16de55ab98 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:20.116338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f616b-9194-49ea-94d4-cf16de55ab98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:20.116605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f616b-9194-49ea-94d4-cf16de55ab98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:20.117276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f616b-9194-49ea-94d4-cf16de55ab98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:20.118858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f616b-9194-49ea-94d4-cf16de55ab98 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:20.130289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 920/1869] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:20.134563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa3d2b-ad9f-41e8-b45a-9435d4911e78 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:20.144347 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaa3d2b-ad9f-41e8-b45a-9435d4911e78 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:20.148822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11378de4-37c0-43cb-99bc-9020e326b80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:20.149412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11378de4-37c0-43cb-99bc-9020e326b80e admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:20.149688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11378de4-37c0-43cb-99bc-9020e326b80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:20.149960 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11378de4-37c0-43cb-99bc-9020e326b80e admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:20.150591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11378de4-37c0-43cb-99bc-9020e326b80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:20.152207 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11378de4-37c0-43cb-99bc-9020e326b80e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:20.162286 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 950/1870] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:20.166136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f616b-9194-49ea-94d4-cf16de55ab98 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:20.176220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f616b-9194-49ea-94d4-cf16de55ab98 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:20.181449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d72d1-6473-4fa9-95a3-575969860142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:20.182088 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d72d1-6473-4fa9-95a3-575969860142 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:20.182368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d72d1-6473-4fa9-95a3-575969860142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:20.182643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d72d1-6473-4fa9-95a3-575969860142 admin admin] PATH_INFO: `/v3/projects/b1697b1ad2a64de38ec7edcd2f3e4a6d/users/04287d76ff85450a8ea2d27ded8ab6b0/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:20.197181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d72d1-6473-4fa9-95a3-575969860142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1697b1ad2a64de38ec7edcd2f3e4a6d, user_id=04287d76ff85450a8ea2d27ded8ab6b0, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:20.199877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825d72d1-6473-4fa9-95a3-575969860142 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:20.290234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 921/1871] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:20 2022] PUT /identity/v3/projects/b1697b1ad2a64de38ec7edcd2f3e4a6d/users/04287d76ff85450a8ea2d27ded8ab6b0/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:20.302147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5adfb3-2648-4b9a-a178-73ac3098d31b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:20.302432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5adfb3-2648-4b9a-a178-73ac3098d31b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:20.302750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5adfb3-2648-4b9a-a178-73ac3098d31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:20.329951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-2d5adfb3-2648-4b9a-a178-73ac3098d31b None None] MFA Rules not processed for user `04287d76ff85450a8ea2d27ded8ab6b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:20.331173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2d5adfb3-2648-4b9a-a178-73ac3098d31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:20.400652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 951/1872] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:20.405195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d72d1-6473-4fa9-95a3-575969860142 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:20.466290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d72d1-6473-4fa9-95a3-575969860142 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:20.473699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b08375-8302-44d2-906b-bff8204b44c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:20.474396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b08375-8302-44d2-906b-bff8204b44c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:20.474712 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b08375-8302-44d2-906b-bff8204b44c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:20.474987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b08375-8302-44d2-906b-bff8204b44c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:20.475638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b08375-8302-44d2-906b-bff8204b44c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:20.477611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b08375-8302-44d2-906b-bff8204b44c8 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:20.578063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 922/1873] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:20 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:20.584496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5adfb3-2648-4b9a-a178-73ac3098d31b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:20.595706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5adfb3-2648-4b9a-a178-73ac3098d31b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:20.651055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d955a-187f-40d7-8cd1-621328b35501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:20.651730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d955a-187f-40d7-8cd1-621328b35501 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:20.652037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d955a-187f-40d7-8cd1-621328b35501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:20.652306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d955a-187f-40d7-8cd1-621328b35501 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:20.652946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d955a-187f-40d7-8cd1-621328b35501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:20.654844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d955a-187f-40d7-8cd1-621328b35501 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:20.724732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 952/1874] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:20 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:20.728834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b08375-8302-44d2-906b-bff8204b44c8 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:20.739520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b08375-8302-44d2-906b-bff8204b44c8 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:20.744448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b387e-691e-45da-921a-727635b70d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:20.745166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b387e-691e-45da-921a-727635b70d76 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:20.745511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b387e-691e-45da-921a-727635b70d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:20.745854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b387e-691e-45da-921a-727635b70d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:20.746608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b387e-691e-45da-921a-727635b70d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:20.748348 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b387e-691e-45da-921a-727635b70d76 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:20.759359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 923/1875] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:20.763391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d955a-187f-40d7-8cd1-621328b35501 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:20.774140 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d955a-187f-40d7-8cd1-621328b35501 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:20.778778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e848-7be4-4f59-8724-d9c0b76c5bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:20.779376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e848-7be4-4f59-8724-d9c0b76c5bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:20.787651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e848-7be4-4f59-8724-d9c0b76c5bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:20.787651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e848-7be4-4f59-8724-d9c0b76c5bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:20.787651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55e848-7be4-4f59-8724-d9c0b76c5bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:20.787651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55e848-7be4-4f59-8724-d9c0b76c5bf9 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:20.792846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 953/1876] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:20.797373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b387e-691e-45da-921a-727635b70d76 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:20.807522 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b387e-691e-45da-921a-727635b70d76 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:20.812644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd42240-deda-481e-a5f7-799ae36cfd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:20.813309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd42240-deda-481e-a5f7-799ae36cfd92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:20.813593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd42240-deda-481e-a5f7-799ae36cfd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:20.816356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd42240-deda-481e-a5f7-799ae36cfd92 admin admin] PATH_INFO: `/v3/projects/696aedebdc3b49b890ce0c62a85ee028/users/677da02f76e94604ba337004eeac53e9/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:20.829252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd42240-deda-481e-a5f7-799ae36cfd92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=696aedebdc3b49b890ce0c62a85ee028, user_id=677da02f76e94604ba337004eeac53e9, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:20.832015 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd42240-deda-481e-a5f7-799ae36cfd92 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:20.917482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:20.922547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 924/1877] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:20 2022] PUT /identity/v3/projects/696aedebdc3b49b890ce0c62a85ee028/users/677da02f76e94604ba337004eeac53e9/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:20.932956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347312e5-515e-4996-98c4-b6fac86fbe12 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:20.933247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347312e5-515e-4996-98c4-b6fac86fbe12 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:20.933528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347312e5-515e-4996-98c4-b6fac86fbe12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:20.962070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-347312e5-515e-4996-98c4-b6fac86fbe12 None None] MFA Rules not processed for user `677da02f76e94604ba337004eeac53e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:20.963279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-347312e5-515e-4996-98c4-b6fac86fbe12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:20.967280 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:21.028010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 954/1878] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:21.032559 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd42240-deda-481e-a5f7-799ae36cfd92 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:21.095070 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd42240-deda-481e-a5f7-799ae36cfd92 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:21.099625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13053bfd-a46b-4aa2-8d46-8ac07340c5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:21.100289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13053bfd-a46b-4aa2-8d46-8ac07340c5ff admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:21.100600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13053bfd-a46b-4aa2-8d46-8ac07340c5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:21.100830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13053bfd-a46b-4aa2-8d46-8ac07340c5ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:21.101465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13053bfd-a46b-4aa2-8d46-8ac07340c5ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:21.103389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13053bfd-a46b-4aa2-8d46-8ac07340c5ff admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:21.177172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 925/1879] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:21 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:21.181337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347312e5-515e-4996-98c4-b6fac86fbe12 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:21.192982 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347312e5-515e-4996-98c4-b6fac86fbe12 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:21.260288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722ad33-ddd4-46bd-b414-2e343c3f645b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:21.262379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722ad33-ddd4-46bd-b414-2e343c3f645b admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:21.262379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722ad33-ddd4-46bd-b414-2e343c3f645b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:21.262379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a722ad33-ddd4-46bd-b414-2e343c3f645b admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:21.262379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722ad33-ddd4-46bd-b414-2e343c3f645b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:21.264524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a722ad33-ddd4-46bd-b414-2e343c3f645b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:21.341243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 955/1880] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:21 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:21.345276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13053bfd-a46b-4aa2-8d46-8ac07340c5ff admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:21.359043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13053bfd-a46b-4aa2-8d46-8ac07340c5ff admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:21.367234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfe523-b78d-4499-9c8c-b88a33ca53e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:21.367881 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfe523-b78d-4499-9c8c-b88a33ca53e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:21.368165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfe523-b78d-4499-9c8c-b88a33ca53e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:21.368442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfe523-b78d-4499-9c8c-b88a33ca53e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:21.369089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfe523-b78d-4499-9c8c-b88a33ca53e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:21.370686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfe523-b78d-4499-9c8c-b88a33ca53e6 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:21.382140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 926/1881] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:21.386652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722ad33-ddd4-46bd-b414-2e343c3f645b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:21.396566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a722ad33-ddd4-46bd-b414-2e343c3f645b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:21.401173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba2bbe-5245-4191-ab32-7f560c58e293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:21.401817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2bbe-5245-4191-ab32-7f560c58e293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:21.402095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2bbe-5245-4191-ab32-7f560c58e293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:21.402370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba2bbe-5245-4191-ab32-7f560c58e293 admin admin] PATH_INFO: `/v3/projects/c386e8cf1fd6429f9eb70ace7baa1c3a/users/17c3e0ac3f014897bce65fdac4200ef1/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:21.417660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba2bbe-5245-4191-ab32-7f560c58e293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c386e8cf1fd6429f9eb70ace7baa1c3a, user_id=17c3e0ac3f014897bce65fdac4200ef1, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:21.420336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba2bbe-5245-4191-ab32-7f560c58e293 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:21.487690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 956/1882] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:21 2022] PUT /identity/v3/projects/c386e8cf1fd6429f9eb70ace7baa1c3a/users/17c3e0ac3f014897bce65fdac4200ef1/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:21.492385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfe523-b78d-4499-9c8c-b88a33ca53e6 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:21.503346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfe523-b78d-4499-9c8c-b88a33ca53e6 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:21.563824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abad3a7-838a-4ff0-a497-7c2a8f0d32f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:21.564430 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abad3a7-838a-4ff0-a497-7c2a8f0d32f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:21.564710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abad3a7-838a-4ff0-a497-7c2a8f0d32f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:21.564981 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abad3a7-838a-4ff0-a497-7c2a8f0d32f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:21.565618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abad3a7-838a-4ff0-a497-7c2a8f0d32f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:21.567908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abad3a7-838a-4ff0-a497-7c2a8f0d32f2 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:21.588430 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 927/1883] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:21.588560 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba2bbe-5245-4191-ab32-7f560c58e293 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:21.594619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba2bbe-5245-4191-ab32-7f560c58e293 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:21.599253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b8b85-e5e2-4451-9829-7fb6990be947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:21.600069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b8b85-e5e2-4451-9829-7fb6990be947 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:21.600496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b8b85-e5e2-4451-9829-7fb6990be947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:21.600774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b8b85-e5e2-4451-9829-7fb6990be947 admin admin] PATH_INFO: `/v3/projects/c386e8cf1fd6429f9eb70ace7baa1c3a/users/17c3e0ac3f014897bce65fdac4200ef1/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:21.602657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b8b85-e5e2-4451-9829-7fb6990be947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c386e8cf1fd6429f9eb70ace7baa1c3a, user_id=17c3e0ac3f014897bce65fdac4200ef1, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:21.605403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b8b85-e5e2-4451-9829-7fb6990be947 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:21.617422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 957/1884] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:21 2022] PUT /identity/v3/projects/c386e8cf1fd6429f9eb70ace7baa1c3a/users/17c3e0ac3f014897bce65fdac4200ef1/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:21.628832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980dfb3-7f96-45ee-80d8-c84cdc157093 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:21.629125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980dfb3-7f96-45ee-80d8-c84cdc157093 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:21.629979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980dfb3-7f96-45ee-80d8-c84cdc157093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:21.658187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0980dfb3-7f96-45ee-80d8-c84cdc157093 None None] MFA Rules not processed for user `17c3e0ac3f014897bce65fdac4200ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:21.659401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0980dfb3-7f96-45ee-80d8-c84cdc157093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:21.731222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:21.741349 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 928/1885] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:21.753678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b8b85-e5e2-4451-9829-7fb6990be947 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:21.755897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0e9b8b85-e5e2-4451-9829-7fb6990be947 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:21.826194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b8b85-e5e2-4451-9829-7fb6990be947 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:21.830101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea0b0c-d141-43bd-aac1-70d2cc97e30b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:21.830711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea0b0c-d141-43bd-aac1-70d2cc97e30b service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:21.831260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea0b0c-d141-43bd-aac1-70d2cc97e30b service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:21.831648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ea0b0c-d141-43bd-aac1-70d2cc97e30b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:21.841228 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:21.888665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea0b0c-d141-43bd-aac1-70d2cc97e30b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:21.890154 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ea0b0c-d141-43bd-aac1-70d2cc97e30b service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:21.899167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 958/1886] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:23:22.136714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980dfb3-7f96-45ee-80d8-c84cdc157093 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:22.138608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0980dfb3-7f96-45ee-80d8-c84cdc157093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:22.151699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980dfb3-7f96-45ee-80d8-c84cdc157093 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:22.155408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb41561-fe9f-407b-ade2-9119c2212b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:22.156051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41561-fe9f-407b-ade2-9119c2212b7b service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:22.156332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41561-fe9f-407b-ade2-9119c2212b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:22.156604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb41561-fe9f-407b-ade2-9119c2212b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:22.158921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-edb41561-fe9f-407b-ade2-9119c2212b7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:22.226273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb41561-fe9f-407b-ade2-9119c2212b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:22.227749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb41561-fe9f-407b-ade2-9119c2212b7b service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:22.237933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 929/1887] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:23:25.086502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea0b0c-d141-43bd-aac1-70d2cc97e30b service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:25.096267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ea0b0c-d141-43bd-aac1-70d2cc97e30b service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:25.100557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03c52b-0f92-45c4-9c40-7954d7a664ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:25.101166 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03c52b-0f92-45c4-9c40-7954d7a664ae service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:25.101446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03c52b-0f92-45c4-9c40-7954d7a664ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:25.101719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03c52b-0f92-45c4-9c40-7954d7a664ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:25.164007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03c52b-0f92-45c4-9c40-7954d7a664ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:25.165510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03c52b-0f92-45c4-9c40-7954d7a664ae service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:25.175738 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 959/1888] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:23:25.521833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb41561-fe9f-407b-ade2-9119c2212b7b service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:25.523709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-edb41561-fe9f-407b-ade2-9119c2212b7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:25.604967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb41561-fe9f-407b-ade2-9119c2212b7b service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:25.609682 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:25.610317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:25.610598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:25.610871 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] PATH_INFO: `/v3/users/80c9b81ca37549729dee71c163a8819c` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:25.612610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80c9b81ca37549729dee71c163a8819c)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:25.614713 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:25.814986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80c9b81ca37549729dee71c163a8819c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:25.886948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80c9b81ca37549729dee71c163a8819c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:25.887908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80c9b81ca37549729dee71c163a8819c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:25.888654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80c9b81ca37549729dee71c163a8819c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:25.896913 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80c9b81ca37549729dee71c163a8819c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:25.915799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 930/1889] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:25 2022] DELETE /identity/v3/users/80c9b81ca37549729dee71c163a8819c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:25.921058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdcfd-5c71-480f-9530-02387d2ec1d0 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:25.922049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdcfd-5c71-480f-9530-02387d2ec1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:25.922814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fdcfd-5c71-480f-9530-02387d2ec1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:25.945711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-299fdcfd-5c71-480f-9530-02387d2ec1d0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:25.947954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-299fdcfd-5c71-480f-9530-02387d2ec1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:26.008568 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 960/1890] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:26.018820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:26.023794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:26.031394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:26.082732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3936a16-f78f-4b0d-9e7d-852ac70743b0 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:26.087719 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c1bee-7cc3-45f3-9eea-97b2bc0dc099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:26.094994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c1bee-7cc3-45f3-9eea-97b2bc0dc099 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:26.094994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c1bee-7cc3-45f3-9eea-97b2bc0dc099 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:26.094994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c1bee-7cc3-45f3-9eea-97b2bc0dc099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:26.098514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c1bee-7cc3-45f3-9eea-97b2bc0dc099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:26.100055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c1bee-7cc3-45f3-9eea-97b2bc0dc099 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:26.109777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 931/1891] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:23:26.702986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fdcfd-5c71-480f-9530-02387d2ec1d0 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:26.704872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-299fdcfd-5c71-480f-9530-02387d2ec1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:26.726207 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fdcfd-5c71-480f-9530-02387d2ec1d0 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:26.731442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:26.732192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:26.732466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:26.732738 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] PATH_INFO: `/v3/projects/a4ba5a53fa8e480e88d4e953a11ceae0` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:26.733882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4ba5a53fa8e480e88d4e953a11ceae0)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:26.736146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:26.835431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:26.838240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4ba5a53fa8e480e88d4e953a11ceae0', 'request_id': 'req-631ba531-fa66-49f7-bcb8-b0534294bf00'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:26.843744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4ba5a53fa8e480e88d4e953a11ceae0', 'request_id': 'req-631ba531-fa66-49f7-bcb8-b0534294bf00'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:26.906281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4ba5a53fa8e480e88d4e953a11ceae0', 'request_id': 'req-631ba531-fa66-49f7-bcb8-b0534294bf00'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:26.919324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] The token cache is being invalidate because project a4ba5a53fa8e480e88d4e953a11ceae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:26.919905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:26.920646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:26.922636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 961/1892] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:26 2022] DELETE /identity/v3/projects/a4ba5a53fa8e480e88d4e953a11ceae0 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:26.926587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c1bee-7cc3-45f3-9eea-97b2bc0dc099 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:26.936294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c1bee-7cc3-45f3-9eea-97b2bc0dc099 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:27.009004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a94edf-8064-480c-86ff-9794ab3cbcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:27.009665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a94edf-8064-480c-86ff-9794ab3cbcb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:27.009946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a94edf-8064-480c-86ff-9794ab3cbcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:27.010222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a94edf-8064-480c-86ff-9794ab3cbcb7 admin admin] PATH_INFO: `/v3/users/e16c844eaea5479aa1c16c1debbd8016` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:27.013978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a94edf-8064-480c-86ff-9794ab3cbcb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e16c844eaea5479aa1c16c1debbd8016)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:27.015973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a94edf-8064-480c-86ff-9794ab3cbcb7 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:27.243015 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-59a94edf-8064-480c-86ff-9794ab3cbcb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e16c844eaea5479aa1c16c1debbd8016'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:27.306585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-59a94edf-8064-480c-86ff-9794ab3cbcb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e16c844eaea5479aa1c16c1debbd8016'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:27.307243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-59a94edf-8064-480c-86ff-9794ab3cbcb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e16c844eaea5479aa1c16c1debbd8016'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:27.308009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-59a94edf-8064-480c-86ff-9794ab3cbcb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e16c844eaea5479aa1c16c1debbd8016'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:27.315722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-59a94edf-8064-480c-86ff-9794ab3cbcb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e16c844eaea5479aa1c16c1debbd8016'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:27.335844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 932/1893] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:26 2022] DELETE /identity/v3/users/e16c844eaea5479aa1c16c1debbd8016 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:27.821314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:27.821933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:27.837021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ba531-fa66-49f7-bcb8-b0534294bf00 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:27.897269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:27.897912 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:27.898192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:27.898471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] PATH_INFO: `/v3/projects/657d417959354d3fb1a409a77d58c414` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:27.899596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=657d417959354d3fb1a409a77d58c414)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:27.901535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:28.036987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657d417959354d3fb1a409a77d58c414', 'request_id': 'req-c5331a70-629b-42ad-a4ab-0bc6b67762f5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:28.042727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657d417959354d3fb1a409a77d58c414', 'request_id': 'req-c5331a70-629b-42ad-a4ab-0bc6b67762f5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:28.106480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657d417959354d3fb1a409a77d58c414', 'request_id': 'req-c5331a70-629b-42ad-a4ab-0bc6b67762f5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:28.118061 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] The token cache is being invalidate because project 657d417959354d3fb1a409a77d58c414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:28.118713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:28.118967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c5331a70-629b-42ad-a4ab-0bc6b67762f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:28.120528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 962/1894] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:27 2022] DELETE /identity/v3/projects/657d417959354d3fb1a409a77d58c414 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:28.125606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a42039-545a-4f59-bfe3-bf515f27efea None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:28.126125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a42039-545a-4f59-bfe3-bf515f27efea None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:28.126228 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a42039-545a-4f59-bfe3-bf515f27efea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:28.145797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-61a42039-545a-4f59-bfe3-bf515f27efea None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:28.146982 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-61a42039-545a-4f59-bfe3-bf515f27efea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:28.173322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:28.207366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 933/1895] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:28.215082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b36ce-f539-4e07-8a2d-8eaeb2e12a1c None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:28.216232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b36ce-f539-4e07-8a2d-8eaeb2e12a1c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:28.216232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b36ce-f539-4e07-8a2d-8eaeb2e12a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:28.235638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-192b36ce-f539-4e07-8a2d-8eaeb2e12a1c None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:28.236808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-192b36ce-f539-4e07-8a2d-8eaeb2e12a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:28.244602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 963/1896] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:28.248914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a42039-545a-4f59-bfe3-bf515f27efea None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:28.258335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a42039-545a-4f59-bfe3-bf515f27efea None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:28.262952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a77527-32e3-44a9-a91f-91c83a846f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:28.263540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a77527-32e3-44a9-a91f-91c83a846f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:28.263845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a77527-32e3-44a9-a91f-91c83a846f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:28.264118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a77527-32e3-44a9-a91f-91c83a846f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:28.264794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a77527-32e3-44a9-a91f-91c83a846f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:23:28.265292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a77527-32e3-44a9-a91f-91c83a846f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:28.267217 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a77527-32e3-44a9-a91f-91c83a846f48 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:28.279411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 934/1897] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:23:28.283490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b36ce-f539-4e07-8a2d-8eaeb2e12a1c None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:28.293215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b36ce-f539-4e07-8a2d-8eaeb2e12a1c None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:28.298079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59113f4c-cd5d-417c-b4b8-fca9d9cdadf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:28.298683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113f4c-cd5d-417c-b4b8-fca9d9cdadf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:28.298966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113f4c-cd5d-417c-b4b8-fca9d9cdadf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:28.299242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59113f4c-cd5d-417c-b4b8-fca9d9cdadf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:28.299902 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59113f4c-cd5d-417c-b4b8-fca9d9cdadf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:28.302326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59113f4c-cd5d-417c-b4b8-fca9d9cdadf3 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:28.399591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 964/1898] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:28 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:28.403834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a77527-32e3-44a9-a91f-91c83a846f48 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:28.414959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a77527-32e3-44a9-a91f-91c83a846f48 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:28.470768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844816d-3411-4694-b303-b39d3e4cc926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:28.471366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844816d-3411-4694-b303-b39d3e4cc926 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:28.471647 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844816d-3411-4694-b303-b39d3e4cc926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:28.471952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844816d-3411-4694-b303-b39d3e4cc926 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:28.473083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844816d-3411-4694-b303-b39d3e4cc926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:28.474982 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c844816d-3411-4694-b303-b39d3e4cc926 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:28.549351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 935/1899] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:28 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:28.554032 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59113f4c-cd5d-417c-b4b8-fca9d9cdadf3 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:28.564365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59113f4c-cd5d-417c-b4b8-fca9d9cdadf3 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:28.568987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1979f1-b115-4fe6-8556-bf6bf194251b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:28.569577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1979f1-b115-4fe6-8556-bf6bf194251b admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:28.569856 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1979f1-b115-4fe6-8556-bf6bf194251b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:28.570129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1979f1-b115-4fe6-8556-bf6bf194251b admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:28.570761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1979f1-b115-4fe6-8556-bf6bf194251b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:28.572385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1979f1-b115-4fe6-8556-bf6bf194251b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:28.582258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 965/1900] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:28.586009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844816d-3411-4694-b303-b39d3e4cc926 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:28.595683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844816d-3411-4694-b303-b39d3e4cc926 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:28.600706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270d1688-1b75-48ab-b717-01ebec856b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:28.601288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d1688-1b75-48ab-b717-01ebec856b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:28.601564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d1688-1b75-48ab-b717-01ebec856b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:28.601836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270d1688-1b75-48ab-b717-01ebec856b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:28.602466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d1688-1b75-48ab-b717-01ebec856b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:28.604065 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270d1688-1b75-48ab-b717-01ebec856b8f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:28.618264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 936/1901] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:28.619345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1979f1-b115-4fe6-8556-bf6bf194251b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:28.629501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1979f1-b115-4fe6-8556-bf6bf194251b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:28.634036 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d9404-dff7-4e68-9ad4-ff531d671213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:28.634675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9404-dff7-4e68-9ad4-ff531d671213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:28.634958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9404-dff7-4e68-9ad4-ff531d671213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:28.635232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d9404-dff7-4e68-9ad4-ff531d671213 admin admin] PATH_INFO: `/v3/projects/788eaa72876841db8c922aaf0a0750d8/users/dcdb639f67fe4a938597f210f7373a3f/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:28.650453 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d9404-dff7-4e68-9ad4-ff531d671213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=788eaa72876841db8c922aaf0a0750d8, user_id=dcdb639f67fe4a938597f210f7373a3f, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:28.653135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d9404-dff7-4e68-9ad4-ff531d671213 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:28.717294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:28.729412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 966/1902] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:28 2022] PUT /identity/v3/projects/788eaa72876841db8c922aaf0a0750d8/users/dcdb639f67fe4a938597f210f7373a3f/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:28.734578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e284c7d-9dd5-4413-9b59-ce3269aba67c None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:28.734861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e284c7d-9dd5-4413-9b59-ce3269aba67c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:28.735134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e284c7d-9dd5-4413-9b59-ce3269aba67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:28.765304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7e284c7d-9dd5-4413-9b59-ce3269aba67c None None] MFA Rules not processed for user `dcdb639f67fe4a938597f210f7373a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:28.766571 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7e284c7d-9dd5-4413-9b59-ce3269aba67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:28.841475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 937/1903] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:28.845582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d9404-dff7-4e68-9ad4-ff531d671213 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:28.917462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d9404-dff7-4e68-9ad4-ff531d671213 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:28.923047 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6effc21-9b15-4150-b158-d0de6d556f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:28.923733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6effc21-9b15-4150-b158-d0de6d556f87 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:28.924046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6effc21-9b15-4150-b158-d0de6d556f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:28.924370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6effc21-9b15-4150-b158-d0de6d556f87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:28.925011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6effc21-9b15-4150-b158-d0de6d556f87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:28.927165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6effc21-9b15-4150-b158-d0de6d556f87 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:29.047494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 967/1904] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:28 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:29.050877 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e284c7d-9dd5-4413-9b59-ce3269aba67c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:29.061399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e284c7d-9dd5-4413-9b59-ce3269aba67c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:29.086316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:29.117685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ea1e-2114-460d-95d6-336a54e5656a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:29.118304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ea1e-2114-460d-95d6-336a54e5656a admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:29.118607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ea1e-2114-460d-95d6-336a54e5656a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:29.118883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624ea1e-2114-460d-95d6-336a54e5656a admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:29.119513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624ea1e-2114-460d-95d6-336a54e5656a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:29.121548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624ea1e-2114-460d-95d6-336a54e5656a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:29.224059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 938/1905] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:29 2022] POST /identity/v3/users => generated 299 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:29.228086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6effc21-9b15-4150-b158-d0de6d556f87 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:29.238594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6effc21-9b15-4150-b158-d0de6d556f87 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:29.243310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf107af-2171-48bd-9bfb-801d2c0266be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:29.243954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf107af-2171-48bd-9bfb-801d2c0266be admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:29.244236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf107af-2171-48bd-9bfb-801d2c0266be admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:29.244530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf107af-2171-48bd-9bfb-801d2c0266be admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:29.245216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf107af-2171-48bd-9bfb-801d2c0266be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:29.246791 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf107af-2171-48bd-9bfb-801d2c0266be admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:29.259072 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 968/1906] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:29.264710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ea1e-2114-460d-95d6-336a54e5656a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:29.290076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624ea1e-2114-460d-95d6-336a54e5656a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:29.303789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d898261-553b-4b2a-a180-b01a4fe05ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:29.304645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d898261-553b-4b2a-a180-b01a4fe05ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:29.304934 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d898261-553b-4b2a-a180-b01a4fe05ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:29.305314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d898261-553b-4b2a-a180-b01a4fe05ac8 admin admin] PATH_INFO: `/v3/projects/380a73d40ee745828bee8a585ecafab7/users/06a1c4bb2925493690c865b325895191/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:29.326130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d898261-553b-4b2a-a180-b01a4fe05ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=380a73d40ee745828bee8a585ecafab7, user_id=06a1c4bb2925493690c865b325895191, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:29.329094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d898261-553b-4b2a-a180-b01a4fe05ac8 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:29.425722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 939/1907] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:29 2022] PUT /identity/v3/projects/380a73d40ee745828bee8a585ecafab7/users/06a1c4bb2925493690c865b325895191/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:29.430567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf107af-2171-48bd-9bfb-801d2c0266be admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:29.447199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf107af-2171-48bd-9bfb-801d2c0266be admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:29.544918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ffc60-91b5-4a44-8a7d-2c926fa2f441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:29.546272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ffc60-91b5-4a44-8a7d-2c926fa2f441 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:29.547098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ffc60-91b5-4a44-8a7d-2c926fa2f441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:29.548365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ffc60-91b5-4a44-8a7d-2c926fa2f441 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:29.549219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ffc60-91b5-4a44-8a7d-2c926fa2f441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:29.553013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ffc60-91b5-4a44-8a7d-2c926fa2f441 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:29.565449 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 969/1908] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:29.570459 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d898261-553b-4b2a-a180-b01a4fe05ac8 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:29.581558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d898261-553b-4b2a-a180-b01a4fe05ac8 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:29.586699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8b03c-34d4-42f8-abdd-d3f68c704f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:29.587500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8b03c-34d4-42f8-abdd-d3f68c704f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:29.587852 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8b03c-34d4-42f8-abdd-d3f68c704f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:29.591573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8b03c-34d4-42f8-abdd-d3f68c704f6d admin admin] PATH_INFO: `/v3/projects/380a73d40ee745828bee8a585ecafab7/users/06a1c4bb2925493690c865b325895191/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:29.591573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8b03c-34d4-42f8-abdd-d3f68c704f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=380a73d40ee745828bee8a585ecafab7, user_id=06a1c4bb2925493690c865b325895191, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:29.595229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c8b03c-34d4-42f8-abdd-d3f68c704f6d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:29.615376 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 940/1909] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:29 2022] PUT /identity/v3/projects/380a73d40ee745828bee8a585ecafab7/users/06a1c4bb2925493690c865b325895191/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:29.632999 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc8bdf-3ab5-47b3-9321-4842fe29494a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:29.633904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc8bdf-3ab5-47b3-9321-4842fe29494a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:29.634382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc8bdf-3ab5-47b3-9321-4842fe29494a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:29.675305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8b03c-34d4-42f8-abdd-d3f68c704f6d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:29.677644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-07c8b03c-34d4-42f8-abdd-d3f68c704f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:29.681288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8acc8bdf-3ab5-47b3-9321-4842fe29494a None None] MFA Rules not processed for user `06a1c4bb2925493690c865b325895191`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:29.682562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8acc8bdf-3ab5-47b3-9321-4842fe29494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:29.750326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8b03c-34d4-42f8-abdd-d3f68c704f6d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:29.754709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cface9-1bc2-49fb-9c43-8551408a2527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:29.755339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cface9-1bc2-49fb-9c43-8551408a2527 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:29.755600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cface9-1bc2-49fb-9c43-8551408a2527 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:29.755965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cface9-1bc2-49fb-9c43-8551408a2527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:29.757851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-89cface9-1bc2-49fb-9c43-8551408a2527 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:29.765495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:29.774540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 970/1910] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:29.824214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cface9-1bc2-49fb-9c43-8551408a2527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:29.825643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cface9-1bc2-49fb-9c43-8551408a2527 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:29.836829 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 941/1911] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:23:29.882212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc8bdf-3ab5-47b3-9321-4842fe29494a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:29.883626 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8acc8bdf-3ab5-47b3-9321-4842fe29494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:29.895074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc8bdf-3ab5-47b3-9321-4842fe29494a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:29.898616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed610673-ef57-4f03-b140-e21141112e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:29.899211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610673-ef57-4f03-b140-e21141112e54 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:29.899503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610673-ef57-4f03-b140-e21141112e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:29.899757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed610673-ef57-4f03-b140-e21141112e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:29.908462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed610673-ef57-4f03-b140-e21141112e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:29.910387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed610673-ef57-4f03-b140-e21141112e54 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:29.919450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 971/1912] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:23:32.232922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cface9-1bc2-49fb-9c43-8551408a2527 service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:32.267608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:32.301477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cface9-1bc2-49fb-9c43-8551408a2527 service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:32.305990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:32.306615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:32.306891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:32.307160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] PATH_INFO: `/v3/users/dcdb639f67fe4a938597f210f7373a3f` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:32.308300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcdb639f67fe4a938597f210f7373a3f)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:32.310191 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:32.499750 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcdb639f67fe4a938597f210f7373a3f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:32.571902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcdb639f67fe4a938597f210f7373a3f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:32.572554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcdb639f67fe4a938597f210f7373a3f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:32.573166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcdb639f67fe4a938597f210f7373a3f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:32.580824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcdb639f67fe4a938597f210f7373a3f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:32.597741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 942/1913] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:32 2022] DELETE /identity/v3/users/dcdb639f67fe4a938597f210f7373a3f => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:32.602255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a11e6-de08-4263-9abb-a9c0c2346aeb None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:32.602756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a11e6-de08-4263-9abb-a9c0c2346aeb None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:32.602849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a11e6-de08-4263-9abb-a9c0c2346aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:32.622815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ea3a11e6-de08-4263-9abb-a9c0c2346aeb None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:32.624100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ea3a11e6-de08-4263-9abb-a9c0c2346aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:32.624383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:32.625778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:32.686378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 972/1914] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:32.692264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251eee6c-7068-43c8-9cf7-af51162f0473 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:32.696649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a11e6-de08-4263-9abb-a9c0c2346aeb None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:32.697129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:32.697792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:32.697985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ea3a11e6-de08-4263-9abb-a9c0c2346aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:32.698090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:32.698331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] PATH_INFO: `/v3/users/f9db2c80376c4104b3b1d6f04dfd6d65` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:32.699474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9db2c80376c4104b3b1d6f04dfd6d65)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:32.701463 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:32.756106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a11e6-de08-4263-9abb-a9c0c2346aeb None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:32.759800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30849f-15ec-4199-ae74-52cdb923dcd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:32.765074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30849f-15ec-4199-ae74-52cdb923dcd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:32.765074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30849f-15ec-4199-ae74-52cdb923dcd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:32.765074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30849f-15ec-4199-ae74-52cdb923dcd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:32.771239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30849f-15ec-4199-ae74-52cdb923dcd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:32.772719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30849f-15ec-4199-ae74-52cdb923dcd0 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:32.782220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 973/1915] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:23:32.883622 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9db2c80376c4104b3b1d6f04dfd6d65'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:32.886280 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:32.935495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9db2c80376c4104b3b1d6f04dfd6d65'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:32.936599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9db2c80376c4104b3b1d6f04dfd6d65'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:32.937385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9db2c80376c4104b3b1d6f04dfd6d65'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:32.945038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9db2c80376c4104b3b1d6f04dfd6d65'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:32.962248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 943/1916] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:32 2022] DELETE /identity/v3/users/f9db2c80376c4104b3b1d6f04dfd6d65 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:32.966694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c6436-c006-40d4-b233-54402fed314e None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:32.966982 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c6436-c006-40d4-b233-54402fed314e None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:32.967258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c6436-c006-40d4-b233-54402fed314e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:32.988748 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-3b3c6436-c006-40d4-b233-54402fed314e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:32.993908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-3b3c6436-c006-40d4-b233-54402fed314e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:33.059727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 974/1917] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:33.069737 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:33.071194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:33.140046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c405b3-5f83-48de-bcd6-98e453f86319 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:33.143723 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531db88-1208-4dbb-8f26-d2dd9e2541a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:33.144386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531db88-1208-4dbb-8f26-d2dd9e2541a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:33.144662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531db88-1208-4dbb-8f26-d2dd9e2541a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:33.144932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531db88-1208-4dbb-8f26-d2dd9e2541a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:33.154333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531db88-1208-4dbb-8f26-d2dd9e2541a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:33.155754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531db88-1208-4dbb-8f26-d2dd9e2541a9 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:33.168805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 944/1918] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:23:33.202425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c6436-c006-40d4-b233-54402fed314e None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:33.206436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-3b3c6436-c006-40d4-b233-54402fed314e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:33.218276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c6436-c006-40d4-b233-54402fed314e None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:33.225463 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:33.226241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:33.226620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:33.227686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] PATH_INFO: `/v3/projects/788eaa72876841db8c922aaf0a0750d8` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:33.229242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=788eaa72876841db8c922aaf0a0750d8)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:33.231377 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:33.256141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:33.389039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '788eaa72876841db8c922aaf0a0750d8', 'request_id': 'req-e73c1721-e50a-48de-9aaa-f856cdfdbf74'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.395130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '788eaa72876841db8c922aaf0a0750d8', 'request_id': 'req-e73c1721-e50a-48de-9aaa-f856cdfdbf74'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.469268 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '788eaa72876841db8c922aaf0a0750d8', 'request_id': 'req-e73c1721-e50a-48de-9aaa-f856cdfdbf74'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.482349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] The token cache is being invalidate because project 788eaa72876841db8c922aaf0a0750d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:33.482694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.483374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.485140 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 975/1919] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:33 2022] DELETE /identity/v3/projects/788eaa72876841db8c922aaf0a0750d8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:33.489296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531db88-1208-4dbb-8f26-d2dd9e2541a9 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:33.504959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531db88-1208-4dbb-8f26-d2dd9e2541a9 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:33.575967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:33.576605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:33.576885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:33.577166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] PATH_INFO: `/v3/users/06a1c4bb2925493690c865b325895191` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:33.578374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06a1c4bb2925493690c865b325895191)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:33.580295 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:33.624900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:33.630527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:33.639640 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c1721-e50a-48de-9aaa-f856cdfdbf74 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:33.644928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:33.645565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:33.645842 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:33.646114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] PATH_INFO: `/v3/projects/fe74614f3bd346b09e90c0a7fb731a92` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:33.647959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe74614f3bd346b09e90c0a7fb731a92)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:33.649881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:33.781717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06a1c4bb2925493690c865b325895191'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.784767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe74614f3bd346b09e90c0a7fb731a92', 'request_id': 'req-0cc3bf90-8504-4df3-8415-2cb22532d988'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.790529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe74614f3bd346b09e90c0a7fb731a92', 'request_id': 'req-0cc3bf90-8504-4df3-8415-2cb22532d988'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.873061 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06a1c4bb2925493690c865b325895191'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.873862 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06a1c4bb2925493690c865b325895191'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.874466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06a1c4bb2925493690c865b325895191'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.882932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06a1c4bb2925493690c865b325895191'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.890211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:33.901757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 945/1920] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:33 2022] DELETE /identity/v3/users/06a1c4bb2925493690c865b325895191 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:33.980325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe74614f3bd346b09e90c0a7fb731a92', 'request_id': 'req-0cc3bf90-8504-4df3-8415-2cb22532d988'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.992408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] The token cache is being invalidate because project fe74614f3bd346b09e90c0a7fb731a92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:33.992775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.993851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:33.995605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 976/1921] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:33 2022] DELETE /identity/v3/projects/fe74614f3bd346b09e90c0a7fb731a92 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:34.000418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:34.002576 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:34.069122 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655b5a2-2c19-4b05-9749-6798ef1ec5e0 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:34.073680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6507a66-007b-4cfd-9259-a0d5e322e75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:34.074307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6507a66-007b-4cfd-9259-a0d5e322e75c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:34.074589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6507a66-007b-4cfd-9259-a0d5e322e75c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:34.074872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6507a66-007b-4cfd-9259-a0d5e322e75c admin admin] PATH_INFO: `/v3/users/2ac809ede871453c9d8371d92b779048` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:34.076025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6507a66-007b-4cfd-9259-a0d5e322e75c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac809ede871453c9d8371d92b779048)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:34.077948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6507a66-007b-4cfd-9259-a0d5e322e75c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:34.303110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d6507a66-007b-4cfd-9259-a0d5e322e75c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac809ede871453c9d8371d92b779048'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:34.376189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:34.377866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:34.392519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d6507a66-007b-4cfd-9259-a0d5e322e75c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac809ede871453c9d8371d92b779048'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:34.393323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d6507a66-007b-4cfd-9259-a0d5e322e75c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac809ede871453c9d8371d92b779048'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:34.394182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d6507a66-007b-4cfd-9259-a0d5e322e75c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac809ede871453c9d8371d92b779048'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:34.407056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d6507a66-007b-4cfd-9259-a0d5e322e75c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac809ede871453c9d8371d92b779048'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:34.426558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 946/1922] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:33 2022] DELETE /identity/v3/users/2ac809ede871453c9d8371d92b779048 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:34.469306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc3bf90-8504-4df3-8415-2cb22532d988 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:34.508277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:34.526150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:34.526785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:34.527066 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:34.527340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] PATH_INFO: `/v3/projects/380a73d40ee745828bee8a585ecafab7` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:34.528465 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=380a73d40ee745828bee8a585ecafab7)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:34.530383 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:34.650542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '380a73d40ee745828bee8a585ecafab7', 'request_id': 'req-54a2b3ce-39ca-492d-959a-f4a6a64925b1'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:34.656040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '380a73d40ee745828bee8a585ecafab7', 'request_id': 'req-54a2b3ce-39ca-492d-959a-f4a6a64925b1'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:34.740663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '380a73d40ee745828bee8a585ecafab7', 'request_id': 'req-54a2b3ce-39ca-492d-959a-f4a6a64925b1'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:34.753969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] The token cache is being invalidate because project 380a73d40ee745828bee8a585ecafab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:34.754256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:34.754909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-54a2b3ce-39ca-492d-959a-f4a6a64925b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:34.756554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 977/1923] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:34 2022] DELETE /identity/v3/projects/380a73d40ee745828bee8a585ecafab7 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:34.761581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b8fb-697e-4f7c-8c4f-83ef40d35d8a None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:34.767037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b8fb-697e-4f7c-8c4f-83ef40d35d8a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:34.767037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859b8fb-697e-4f7c-8c4f-83ef40d35d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:34.783178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0859b8fb-697e-4f7c-8c4f-83ef40d35d8a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:34.784360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0859b8fb-697e-4f7c-8c4f-83ef40d35d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:34.844105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 947/1924] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:34.852044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012598cf-f43b-4353-9a83-38d820970c0a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:34.852332 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012598cf-f43b-4353-9a83-38d820970c0a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:34.852614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012598cf-f43b-4353-9a83-38d820970c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:34.880615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-012598cf-f43b-4353-9a83-38d820970c0a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:34.880615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-012598cf-f43b-4353-9a83-38d820970c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:34.882098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859b8fb-697e-4f7c-8c4f-83ef40d35d8a None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:34.883645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0859b8fb-697e-4f7c-8c4f-83ef40d35d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:34.892167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 978/1925] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:34.893055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012598cf-f43b-4353-9a83-38d820970c0a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:34.897007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859b8fb-697e-4f7c-8c4f-83ef40d35d8a None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:34.912393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:34.912393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:34.912393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:34.912393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] PATH_INFO: `/v3/projects/c75e48ae033d42679c98b796138a08a7` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:34.912924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c75e48ae033d42679c98b796138a08a7)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:34.912924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:34.913073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012598cf-f43b-4353-9a83-38d820970c0a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:34.913629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:34.915659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09b3f1-e4f0-4c63-9ac2-5f44ab0dfbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:34.916341 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b3f1-e4f0-4c63-9ac2-5f44ab0dfbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:34.921785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b3f1-e4f0-4c63-9ac2-5f44ab0dfbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:34.921785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09b3f1-e4f0-4c63-9ac2-5f44ab0dfbc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:34.921785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09b3f1-e4f0-4c63-9ac2-5f44ab0dfbc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:23:34.921785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09b3f1-e4f0-4c63-9ac2-5f44ab0dfbc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:34.921785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09b3f1-e4f0-4c63-9ac2-5f44ab0dfbc2 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:34.940633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 979/1926] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:23:34.945730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09b3f1-e4f0-4c63-9ac2-5f44ab0dfbc2 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:34.955448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09b3f1-e4f0-4c63-9ac2-5f44ab0dfbc2 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:34.960295 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6842943-fec4-49fd-a018-b4f5df93d691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:34.960925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6842943-fec4-49fd-a018-b4f5df93d691 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:34.961245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6842943-fec4-49fd-a018-b4f5df93d691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:34.961514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6842943-fec4-49fd-a018-b4f5df93d691 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:34.962127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6842943-fec4-49fd-a018-b4f5df93d691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:34.964005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6842943-fec4-49fd-a018-b4f5df93d691 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.061300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c75e48ae033d42679c98b796138a08a7', 'request_id': 'req-067e7a08-a183-4887-9f07-e101ac4a6532'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:35.090005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 980/1927] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:34 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:35.094131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6842943-fec4-49fd-a018-b4f5df93d691 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.129160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c75e48ae033d42679c98b796138a08a7', 'request_id': 'req-067e7a08-a183-4887-9f07-e101ac4a6532'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:35.135882 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c75e48ae033d42679c98b796138a08a7', 'request_id': 'req-067e7a08-a183-4887-9f07-e101ac4a6532'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:35.146450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] The token cache is being invalidate because project c75e48ae033d42679c98b796138a08a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:35.146743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:35.147399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-067e7a08-a183-4887-9f07-e101ac4a6532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:35.148937 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 948/1928] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:34 2022] DELETE /identity/v3/projects/c75e48ae033d42679c98b796138a08a7 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:35.153403 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c0525-260c-4819-a3ee-313c50618509 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.153687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c0525-260c-4819-a3ee-313c50618509 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.153964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c0525-260c-4819-a3ee-313c50618509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.156659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6842943-fec4-49fd-a018-b4f5df93d691 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.161776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23602fbd-ef1b-4899-9154-93f164d31f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.162553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23602fbd-ef1b-4899-9154-93f164d31f33 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.162830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23602fbd-ef1b-4899-9154-93f164d31f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.163103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23602fbd-ef1b-4899-9154-93f164d31f33 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.163724 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23602fbd-ef1b-4899-9154-93f164d31f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.165651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23602fbd-ef1b-4899-9154-93f164d31f33 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.174144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-882c0525-260c-4819-a3ee-313c50618509 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:35.178791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-882c0525-260c-4819-a3ee-313c50618509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:35.184069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 949/1929] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:23:35.193064 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d06c52-2253-4aa7-8bf2-94848d74d054 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.193347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d06c52-2253-4aa7-8bf2-94848d74d054 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.193641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d06c52-2253-4aa7-8bf2-94848d74d054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.212998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-15d06c52-2253-4aa7-8bf2-94848d74d054 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:35.214181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-15d06c52-2253-4aa7-8bf2-94848d74d054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:35.222542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 950/1930] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:23:35.226389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d06c52-2253-4aa7-8bf2-94848d74d054 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.234927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d06c52-2253-4aa7-8bf2-94848d74d054 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.240107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05eea2-6c12-4ae3-a75d-508df5524613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.240656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05eea2-6c12-4ae3-a75d-508df5524613 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.240937 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05eea2-6c12-4ae3-a75d-508df5524613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.241207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa05eea2-6c12-4ae3-a75d-508df5524613 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.241871 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05eea2-6c12-4ae3-a75d-508df5524613 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:23:35.242172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05eea2-6c12-4ae3-a75d-508df5524613 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.243734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa05eea2-6c12-4ae3-a75d-508df5524613 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.254913 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 951/1931] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:23:35.258802 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05eea2-6c12-4ae3-a75d-508df5524613 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.261424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 981/1932] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:35 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:35.268468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa05eea2-6c12-4ae3-a75d-508df5524613 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.273123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d25e5-7b84-4231-abe4-2b3502621b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.273251 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23602fbd-ef1b-4899-9154-93f164d31f33 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.273701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d25e5-7b84-4231-abe4-2b3502621b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.273976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d25e5-7b84-4231-abe4-2b3502621b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.274253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d25e5-7b84-4231-abe4-2b3502621b49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.274826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-23602fbd-ef1b-4899-9154-93f164d31f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:35.274951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d25e5-7b84-4231-abe4-2b3502621b49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.276777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d25e5-7b84-4231-abe4-2b3502621b49 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.287327 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23602fbd-ef1b-4899-9154-93f164d31f33 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.292808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e96007-4efc-4780-98b2-b1783a5a1a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.293469 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e96007-4efc-4780-98b2-b1783a5a1a1f admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.293672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e96007-4efc-4780-98b2-b1783a5a1a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.304599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e96007-4efc-4780-98b2-b1783a5a1a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.304599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e96007-4efc-4780-98b2-b1783a5a1a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.304599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e96007-4efc-4780-98b2-b1783a5a1a1f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.311412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 982/1933] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:35.312356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e96007-4efc-4780-98b2-b1783a5a1a1f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.321764 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e96007-4efc-4780-98b2-b1783a5a1a1f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.327585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e069c-a4e2-4dd5-8c46-c61e2cc70c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.327585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e069c-a4e2-4dd5-8c46-c61e2cc70c49 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.327585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e069c-a4e2-4dd5-8c46-c61e2cc70c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.328046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e069c-a4e2-4dd5-8c46-c61e2cc70c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.328597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e069c-a4e2-4dd5-8c46-c61e2cc70c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.330187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e069c-a4e2-4dd5-8c46-c61e2cc70c49 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.341029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 983/1934] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:35.344708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e069c-a4e2-4dd5-8c46-c61e2cc70c49 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.354243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e069c-a4e2-4dd5-8c46-c61e2cc70c49 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.358922 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a41716-bd37-4c0f-8884-f1819e537aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.359534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a41716-bd37-4c0f-8884-f1819e537aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.359845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a41716-bd37-4c0f-8884-f1819e537aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.360119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a41716-bd37-4c0f-8884-f1819e537aa2 admin admin] PATH_INFO: `/v3/projects/c290ec08842142ebaa850da2509d4d45/users/ed6bf61b48004dbf977d93a4c5efcc7d/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.374822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a41716-bd37-4c0f-8884-f1819e537aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c290ec08842142ebaa850da2509d4d45, user_id=ed6bf61b48004dbf977d93a4c5efcc7d, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.377431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a41716-bd37-4c0f-8884-f1819e537aa2 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.404648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 952/1935] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:35 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:35.408430 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d25e5-7b84-4231-abe4-2b3502621b49 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.469753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d25e5-7b84-4231-abe4-2b3502621b49 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.474354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a51523-cb36-4c0b-b9af-39b1f6f12522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.474970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a51523-cb36-4c0b-b9af-39b1f6f12522 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.475245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a51523-cb36-4c0b-b9af-39b1f6f12522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.475515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a51523-cb36-4c0b-b9af-39b1f6f12522 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.476178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a51523-cb36-4c0b-b9af-39b1f6f12522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.478083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a51523-cb36-4c0b-b9af-39b1f6f12522 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.478190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 984/1936] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:35 2022] PUT /identity/v3/projects/c290ec08842142ebaa850da2509d4d45/users/ed6bf61b48004dbf977d93a4c5efcc7d/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:35.489414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164b74b-8e4b-458b-ac6b-339af301593b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.489710 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164b74b-8e4b-458b-ac6b-339af301593b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.489986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164b74b-8e4b-458b-ac6b-339af301593b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.516931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1164b74b-8e4b-458b-ac6b-339af301593b None None] MFA Rules not processed for user `ed6bf61b48004dbf977d93a4c5efcc7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:35.518131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1164b74b-8e4b-458b-ac6b-339af301593b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:35.580550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 953/1937] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:35 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:35.581363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a51523-cb36-4c0b-b9af-39b1f6f12522 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.585281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 985/1938] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:23:35.589547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164b74b-8e4b-458b-ac6b-339af301593b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.591066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a51523-cb36-4c0b-b9af-39b1f6f12522 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.651660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3881a-1965-4d6d-abcb-9c8ce829bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.652302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3881a-1965-4d6d-abcb-9c8ce829bb86 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.652577 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3881a-1965-4d6d-abcb-9c8ce829bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.652674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164b74b-8e4b-458b-ac6b-339af301593b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.653093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3881a-1965-4d6d-abcb-9c8ce829bb86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.653580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3881a-1965-4d6d-abcb-9c8ce829bb86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.655232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3881a-1965-4d6d-abcb-9c8ce829bb86 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.657394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdf6c9-2507-4399-a78f-d2d5a50e1bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.657990 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdf6c9-2507-4399-a78f-d2d5a50e1bbb admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.658267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdf6c9-2507-4399-a78f-d2d5a50e1bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.658594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fdf6c9-2507-4399-a78f-d2d5a50e1bbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.659192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdf6c9-2507-4399-a78f-d2d5a50e1bbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.661131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fdf6c9-2507-4399-a78f-d2d5a50e1bbb admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.665432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 954/1939] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:35.668502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:35.669309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3881a-1965-4d6d-abcb-9c8ce829bb86 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.679572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3881a-1965-4d6d-abcb-9c8ce829bb86 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.684432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a3f7a-8fa3-4fad-90ac-557a66670aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.685029 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a3f7a-8fa3-4fad-90ac-557a66670aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.685314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a3f7a-8fa3-4fad-90ac-557a66670aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.685598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a3f7a-8fa3-4fad-90ac-557a66670aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.686289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a3f7a-8fa3-4fad-90ac-557a66670aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.688946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a3f7a-8fa3-4fad-90ac-557a66670aa1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.700201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 955/1940] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:35.704213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a3f7a-8fa3-4fad-90ac-557a66670aa1 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.713421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a3f7a-8fa3-4fad-90ac-557a66670aa1 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.717953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa5aa2-5f09-403c-896d-748b2a403f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.718582 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa5aa2-5f09-403c-896d-748b2a403f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.718865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa5aa2-5f09-403c-896d-748b2a403f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.719142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa5aa2-5f09-403c-896d-748b2a403f91 admin admin] PATH_INFO: `/v3/projects/fcd02cd0e99d4f7a858200a80f763b1f/users/688a6a8b487844838831116d873330c3/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.733063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa5aa2-5f09-403c-896d-748b2a403f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd02cd0e99d4f7a858200a80f763b1f, user_id=688a6a8b487844838831116d873330c3, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.735684 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa5aa2-5f09-403c-896d-748b2a403f91 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.750561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 986/1941] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:35 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:35.754804 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdf6c9-2507-4399-a78f-d2d5a50e1bbb admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.815269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fdf6c9-2507-4399-a78f-d2d5a50e1bbb admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.820666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b629036-d113-4b78-8b19-3b5f10263fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.821267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b629036-d113-4b78-8b19-3b5f10263fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.821545 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b629036-d113-4b78-8b19-3b5f10263fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.821815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b629036-d113-4b78-8b19-3b5f10263fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.822484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b629036-d113-4b78-8b19-3b5f10263fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.824537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b629036-d113-4b78-8b19-3b5f10263fd1 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.855150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 956/1942] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:35 2022] PUT /identity/v3/projects/fcd02cd0e99d4f7a858200a80f763b1f/users/688a6a8b487844838831116d873330c3/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:35.865844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1df99-e4c1-4c78-b22f-969ddb016f88 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.866124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1df99-e4c1-4c78-b22f-969ddb016f88 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.866397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a1df99-e4c1-4c78-b22f-969ddb016f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.875048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:35.895555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c5a1df99-e4c1-4c78-b22f-969ddb016f88 None None] MFA Rules not processed for user `688a6a8b487844838831116d873330c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:35.896786 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c5a1df99-e4c1-4c78-b22f-969ddb016f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:35.905883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 987/1943] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:35 2022] POST /identity/v3/users => generated 298 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:35.910163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b629036-d113-4b78-8b19-3b5f10263fd1 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.919659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b629036-d113-4b78-8b19-3b5f10263fd1 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:35.972488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 957/1944] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:35.972488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1df99-e4c1-4c78-b22f-969ddb016f88 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:35.980265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b16867-c9e5-460e-b5e1-b32b1958464d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:35.980884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b16867-c9e5-460e-b5e1-b32b1958464d admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:35.981164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b16867-c9e5-460e-b5e1-b32b1958464d admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:35.981439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b16867-c9e5-460e-b5e1-b32b1958464d admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:35.982081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b16867-c9e5-460e-b5e1-b32b1958464d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:35.983697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b16867-c9e5-460e-b5e1-b32b1958464d admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:35.994406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 988/1945] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:35.998579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b16867-c9e5-460e-b5e1-b32b1958464d admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:36.009665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b16867-c9e5-460e-b5e1-b32b1958464d admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.014318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f921c92-24eb-481e-9798-ee08cf7d7c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.014973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f921c92-24eb-481e-9798-ee08cf7d7c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.015255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f921c92-24eb-481e-9798-ee08cf7d7c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.015535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f921c92-24eb-481e-9798-ee08cf7d7c12 admin admin] PATH_INFO: `/v3/projects/f226c630a3a145909ab532f40d0543b7/users/fdc3a9dd1a47415baed4f00867af3273/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.033063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f921c92-24eb-481e-9798-ee08cf7d7c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f226c630a3a145909ab532f40d0543b7, user_id=fdc3a9dd1a47415baed4f00867af3273, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.036341 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a1df99-e4c1-4c78-b22f-969ddb016f88 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.037049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f921c92-24eb-481e-9798-ee08cf7d7c12 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.039232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac2188-de69-49b3-8c25-84e31718e23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.040203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac2188-de69-49b3-8c25-84e31718e23a admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.040203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac2188-de69-49b3-8c25-84e31718e23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.040632 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac2188-de69-49b3-8c25-84e31718e23a admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.041271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac2188-de69-49b3-8c25-84e31718e23a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.043252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac2188-de69-49b3-8c25-84e31718e23a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.100866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 989/1946] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:35 2022] PUT /identity/v3/projects/f226c630a3a145909ab532f40d0543b7/users/fdc3a9dd1a47415baed4f00867af3273/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:36.104551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f921c92-24eb-481e-9798-ee08cf7d7c12 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:36.118661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:36.127724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 958/1947] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:35 2022] POST /identity/v3/projects => generated 379 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:36.129067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac2188-de69-49b3-8c25-84e31718e23a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:36.168290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f921c92-24eb-481e-9798-ee08cf7d7c12 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.195994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:36.201038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac2188-de69-49b3-8c25-84e31718e23a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.206343 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3838d1-fa1d-46c5-8f31-4c106811249d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.206944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3838d1-fa1d-46c5-8f31-4c106811249d admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.207220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3838d1-fa1d-46c5-8f31-4c106811249d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.207488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3838d1-fa1d-46c5-8f31-4c106811249d admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.208144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3838d1-fa1d-46c5-8f31-4c106811249d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.210056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3838d1-fa1d-46c5-8f31-4c106811249d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.223433 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7e9ec-5380-46f3-a0ec-91d7fb8c36e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.224330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7e9ec-5380-46f3-a0ec-91d7fb8c36e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.225185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7e9ec-5380-46f3-a0ec-91d7fb8c36e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.225292 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7e9ec-5380-46f3-a0ec-91d7fb8c36e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.226017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7e9ec-5380-46f3-a0ec-91d7fb8c36e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.227683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7e9ec-5380-46f3-a0ec-91d7fb8c36e7 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.238825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 990/1948] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:36.242775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7e9ec-5380-46f3-a0ec-91d7fb8c36e7 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:36.252063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7e9ec-5380-46f3-a0ec-91d7fb8c36e7 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.256650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5959b-3e86-41f7-8c60-4beb24383a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.257260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5959b-3e86-41f7-8c60-4beb24383a22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.261304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5959b-3e86-41f7-8c60-4beb24383a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.261304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5959b-3e86-41f7-8c60-4beb24383a22 admin admin] PATH_INFO: `/v3/projects/f226c630a3a145909ab532f40d0543b7/users/fdc3a9dd1a47415baed4f00867af3273/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.261304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5959b-3e86-41f7-8c60-4beb24383a22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f226c630a3a145909ab532f40d0543b7, user_id=fdc3a9dd1a47415baed4f00867af3273, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.262267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5959b-3e86-41f7-8c60-4beb24383a22 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.271956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 991/1949] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:36 2022] PUT /identity/v3/projects/f226c630a3a145909ab532f40d0543b7/users/fdc3a9dd1a47415baed4f00867af3273/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 29 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:36.281799 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7b8d9-cdc4-475d-80a6-5c9179f90edc None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.282087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7b8d9-cdc4-475d-80a6-5c9179f90edc None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.282364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7b8d9-cdc4-475d-80a6-5c9179f90edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.310737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8dc7b8d9-cdc4-475d-80a6-5c9179f90edc None None] MFA Rules not processed for user `fdc3a9dd1a47415baed4f00867af3273`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:36.311975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8dc7b8d9-cdc4-475d-80a6-5c9179f90edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:36.316474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 959/1950] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:36 2022] POST /identity/v3/users => generated 296 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:36.324356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3838d1-fa1d-46c5-8f31-4c106811249d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:36.336313 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3838d1-fa1d-46c5-8f31-4c106811249d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.369017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:36.389578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 992/1951] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:36 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:23:36.396397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d00d48-c376-415e-b0c5-57c82b30cf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.397238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d00d48-c376-415e-b0c5-57c82b30cf86 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.397739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d00d48-c376-415e-b0c5-57c82b30cf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.398652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d00d48-c376-415e-b0c5-57c82b30cf86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.399371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d00d48-c376-415e-b0c5-57c82b30cf86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.401116 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d00d48-c376-415e-b0c5-57c82b30cf86 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.401584 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7b8d9-cdc4-475d-80a6-5c9179f90edc None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:36.403023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8dc7b8d9-cdc4-475d-80a6-5c9179f90edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:36.413325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 960/1952] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:36.416465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d00d48-c376-415e-b0c5-57c82b30cf86 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:36.426310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d00d48-c376-415e-b0c5-57c82b30cf86 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.431161 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79dcb1-ea70-4710-813f-b08ac3323d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.431833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79dcb1-ea70-4710-813f-b08ac3323d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.432123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79dcb1-ea70-4710-813f-b08ac3323d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.432399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79dcb1-ea70-4710-813f-b08ac3323d62 admin admin] PATH_INFO: `/v3/projects/4f75e7b1560a4f48ade6f486fbcad9ba/users/f3dba5b892a043c599e5e390327fa0f9/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.447651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79dcb1-ea70-4710-813f-b08ac3323d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f75e7b1560a4f48ade6f486fbcad9ba, user_id=f3dba5b892a043c599e5e390327fa0f9, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.450312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d79dcb1-ea70-4710-813f-b08ac3323d62 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.461939 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7b8d9-cdc4-475d-80a6-5c9179f90edc None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.465635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003ac28-3977-48c2-8844-6821bdf2142b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.466226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003ac28-3977-48c2-8844-6821bdf2142b service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.466500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003ac28-3977-48c2-8844-6821bdf2142b service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.466768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003ac28-3977-48c2-8844-6821bdf2142b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.476285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003ac28-3977-48c2-8844-6821bdf2142b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.477765 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003ac28-3977-48c2-8844-6821bdf2142b service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.487458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 993/1953] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:23:36.517455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 961/1954] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:36 2022] PUT /identity/v3/projects/4f75e7b1560a4f48ade6f486fbcad9ba/users/f3dba5b892a043c599e5e390327fa0f9/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:36.524691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003ac28-3977-48c2-8844-6821bdf2142b service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:36.539704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003ac28-3977-48c2-8844-6821bdf2142b service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.595120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e5ef8-9a70-458c-b06c-36ba75c2f7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.595729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e5ef8-9a70-458c-b06c-36ba75c2f7ab admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.596051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e5ef8-9a70-458c-b06c-36ba75c2f7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.596320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e5ef8-9a70-458c-b06c-36ba75c2f7ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.597543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e5ef8-9a70-458c-b06c-36ba75c2f7ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.599163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e5ef8-9a70-458c-b06c-36ba75c2f7ab admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.608656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 994/1955] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:36.613090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79dcb1-ea70-4710-813f-b08ac3323d62 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:36.623372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79dcb1-ea70-4710-813f-b08ac3323d62 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.632635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634172c-5777-4e81-9220-ac57f2aa49ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.633336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634172c-5777-4e81-9220-ac57f2aa49ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.633618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634172c-5777-4e81-9220-ac57f2aa49ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.633896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634172c-5777-4e81-9220-ac57f2aa49ac admin admin] PATH_INFO: `/v3/projects/4f75e7b1560a4f48ade6f486fbcad9ba/users/f3dba5b892a043c599e5e390327fa0f9/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.636089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634172c-5777-4e81-9220-ac57f2aa49ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f75e7b1560a4f48ade6f486fbcad9ba, user_id=f3dba5b892a043c599e5e390327fa0f9, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.639387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634172c-5777-4e81-9220-ac57f2aa49ac admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.639822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e5ef8-9a70-458c-b06c-36ba75c2f7ab admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:36.645068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:36.652210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 962/1956] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:36 2022] PUT /identity/v3/projects/4f75e7b1560a4f48ade6f486fbcad9ba/users/f3dba5b892a043c599e5e390327fa0f9/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:36.665741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9501b-4612-4c1e-b35c-5fecf1900a2f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.666028 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9501b-4612-4c1e-b35c-5fecf1900a2f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.666304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9501b-4612-4c1e-b35c-5fecf1900a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.698128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e5ef8-9a70-458c-b06c-36ba75c2f7ab admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.699816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-aef9501b-4612-4c1e-b35c-5fecf1900a2f None None] MFA Rules not processed for user `f3dba5b892a043c599e5e390327fa0f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:36.701124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-aef9501b-4612-4c1e-b35c-5fecf1900a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:36.746197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:36.748304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38273947-4825-4f81-8789-58b77d9a3cb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.748910 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38273947-4825-4f81-8789-58b77d9a3cb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.749187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38273947-4825-4f81-8789-58b77d9a3cb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.749482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38273947-4825-4f81-8789-58b77d9a3cb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.771107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 963/1957] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:23:36.784993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9501b-4612-4c1e-b35c-5fecf1900a2f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:36.786363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-aef9501b-4612-4c1e-b35c-5fecf1900a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:36.799526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9501b-4612-4c1e-b35c-5fecf1900a2f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:36.803364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d119e8f-89ae-4e6a-80f3-e6bb08a95a5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:36.804202 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119e8f-89ae-4e6a-80f3-e6bb08a95a5f service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:36.804343 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119e8f-89ae-4e6a-80f3-e6bb08a95a5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:36.804667 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d119e8f-89ae-4e6a-80f3-e6bb08a95a5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:36.812086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38273947-4825-4f81-8789-58b77d9a3cb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.813547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38273947-4825-4f81-8789-58b77d9a3cb3 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.814066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d119e8f-89ae-4e6a-80f3-e6bb08a95a5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:36.815501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d119e8f-89ae-4e6a-80f3-e6bb08a95a5f service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:36.823263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 995/1958] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:23:36.826157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 964/1959] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:23:37.383489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38273947-4825-4f81-8789-58b77d9a3cb3 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:37.455589 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38273947-4825-4f81-8789-58b77d9a3cb3 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:37.461634 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:37.462589 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:37.462872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:37.463151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] PATH_INFO: `/v3/users/ed6bf61b48004dbf977d93a4c5efcc7d` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:37.464830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed6bf61b48004dbf977d93a4c5efcc7d)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:37.466770 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:37.481197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:37.732130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed6bf61b48004dbf977d93a4c5efcc7d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:37.811457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed6bf61b48004dbf977d93a4c5efcc7d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:37.812085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed6bf61b48004dbf977d93a4c5efcc7d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:37.812779 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed6bf61b48004dbf977d93a4c5efcc7d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:37.819893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed6bf61b48004dbf977d93a4c5efcc7d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:37.837133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 996/1960] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:37 2022] DELETE /identity/v3/users/ed6bf61b48004dbf977d93a4c5efcc7d => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:37.847022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406c4ed-71c1-4b46-81a0-6e6852d165f2 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:37.848102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406c4ed-71c1-4b46-81a0-6e6852d165f2 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:37.848218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406c4ed-71c1-4b46-81a0-6e6852d165f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:37.867073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0406c4ed-71c1-4b46-81a0-6e6852d165f2 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:37.868321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0406c4ed-71c1-4b46-81a0-6e6852d165f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:37.929310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 965/1961] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:23:37.942154 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:37.943692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:37.943860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406c4ed-71c1-4b46-81a0-6e6852d165f2 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:37.945477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0406c4ed-71c1-4b46-81a0-6e6852d165f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:38.003539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406c4ed-71c1-4b46-81a0-6e6852d165f2 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:38.007632 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0f414-48ef-4169-93fd-dc682d5c65c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:38.008408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0f414-48ef-4169-93fd-dc682d5c65c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:38.008792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0f414-48ef-4169-93fd-dc682d5c65c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:38.009200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0f414-48ef-4169-93fd-dc682d5c65c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:38.010126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e036e20-00b1-4df7-99cc-2668f795a680 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:38.014940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282a157-3d50-4186-afb9-8a85c4fc9b00 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3dba5b892a043c599e5e390327fa0f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f75e7b1560a4f48ade6f486fbcad9ba', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:38.016460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282a157-3d50-4186-afb9-8a85c4fc9b00 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:38.016761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282a157-3d50-4186-afb9-8a85c4fc9b00 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:38.017042 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282a157-3d50-4186-afb9-8a85c4fc9b00 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:38.017706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282a157-3d50-4186-afb9-8a85c4fc9b00 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:23:38.018020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282a157-3d50-4186-afb9-8a85c4fc9b00 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:38.019034 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0f414-48ef-4169-93fd-dc682d5c65c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:38.020214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282a157-3d50-4186-afb9-8a85c4fc9b00 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:38.021022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0f414-48ef-4169-93fd-dc682d5c65c7 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:38.030508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 966/1962] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:23:38.032218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 997/1963] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:23:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:23:38.037075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0f414-48ef-4169-93fd-dc682d5c65c7 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:38.048144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0f414-48ef-4169-93fd-dc682d5c65c7 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:38.053599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be89329-8af3-4ef7-9ce9-c47e75f91afa tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3dba5b892a043c599e5e390327fa0f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f75e7b1560a4f48ade6f486fbcad9ba', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:38.054210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89329-8af3-4ef7-9ce9-c47e75f91afa tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:38.059437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89329-8af3-4ef7-9ce9-c47e75f91afa tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:38.059437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89329-8af3-4ef7-9ce9-c47e75f91afa tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:38.059437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be89329-8af3-4ef7-9ce9-c47e75f91afa tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:38.059437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be89329-8af3-4ef7-9ce9-c47e75f91afa tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:38.164777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 967/1964] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:38 2022] POST /identity/v3/projects => generated 353 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:38.168993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282a157-3d50-4186-afb9-8a85c4fc9b00 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:38.179296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282a157-3d50-4186-afb9-8a85c4fc9b00 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:38.235596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32abe9-6f4c-4d62-a9df-bcbb8ce556fe tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3dba5b892a043c599e5e390327fa0f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f75e7b1560a4f48ade6f486fbcad9ba', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:38.236307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32abe9-6f4c-4d62-a9df-bcbb8ce556fe tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:38.242208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32abe9-6f4c-4d62-a9df-bcbb8ce556fe tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:38.242208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32abe9-6f4c-4d62-a9df-bcbb8ce556fe tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:38.242208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32abe9-6f4c-4d62-a9df-bcbb8ce556fe tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:23:38.242208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32abe9-6f4c-4d62-a9df-bcbb8ce556fe tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:38.242208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32abe9-6f4c-4d62-a9df-bcbb8ce556fe tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:38.251628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 998/1965] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:23:38.295746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be89329-8af3-4ef7-9ce9-c47e75f91afa tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:38.309355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be89329-8af3-4ef7-9ce9-c47e75f91afa tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:38.315591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3dba5b892a043c599e5e390327fa0f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f75e7b1560a4f48ade6f486fbcad9ba', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:38.316964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:38.317391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:38.317812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] PATH_INFO: `/v3/projects/88893212b12f4b42bf766be7cd0ecc85` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:38.319182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: Authorizing `identity:delete_project(project_id=88893212b12f4b42bf766be7cd0ecc85)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:38.321365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:38.418305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:38.421748 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88893212b12f4b42bf766be7cd0ecc85', 'request_id': 'req-54a61346-f0d1-4984-826f-1f442741c6e1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:38.482957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88893212b12f4b42bf766be7cd0ecc85', 'request_id': 'req-54a61346-f0d1-4984-826f-1f442741c6e1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:38.489770 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32abe9-6f4c-4d62-a9df-bcbb8ce556fe tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:38.490910 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88893212b12f4b42bf766be7cd0ecc85', 'request_id': 'req-54a61346-f0d1-4984-826f-1f442741c6e1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:38.491502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0d32abe9-6f4c-4d62-a9df-bcbb8ce556fe tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:38.520335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:38.522091 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] The token cache is being invalidate because project 88893212b12f4b42bf766be7cd0ecc85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:38.522387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:38.523074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:38.529001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 968/1966] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:38 2022] DELETE /identity/v3/projects/88893212b12f4b42bf766be7cd0ecc85 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:38.571137 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32abe9-6f4c-4d62-a9df-bcbb8ce556fe tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:38.629325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:38.629978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:38.630254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:38.630529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] PATH_INFO: `/v3/projects/c290ec08842142ebaa850da2509d4d45` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:38.631784 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c290ec08842142ebaa850da2509d4d45)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:38.633703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:38.756929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c290ec08842142ebaa850da2509d4d45', 'request_id': 'req-370b6750-2ea5-4d00-8571-db91c5f47ce6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:38.762283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c290ec08842142ebaa850da2509d4d45', 'request_id': 'req-370b6750-2ea5-4d00-8571-db91c5f47ce6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:38.852350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c290ec08842142ebaa850da2509d4d45', 'request_id': 'req-370b6750-2ea5-4d00-8571-db91c5f47ce6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:38.864670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] The token cache is being invalidate because project c290ec08842142ebaa850da2509d4d45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:38.864963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:38.865609 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:38.867158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 999/1967] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:38 2022] DELETE /identity/v3/projects/c290ec08842142ebaa850da2509d4d45 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:38.871079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:38.872510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:38.886708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a61346-f0d1-4984-826f-1f442741c6e1 tempest-VolumeQuotaClassesTest-529703013 tempest-VolumeQuotaClassesTest-529703013-project-admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:38.942729 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:38.943360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:38.943647 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:38.943954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] PATH_INFO: `/v3/users/fdc3a9dd1a47415baed4f00867af3273` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:38.945103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdc3a9dd1a47415baed4f00867af3273)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:38.947021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:39.178926 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdc3a9dd1a47415baed4f00867af3273'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.280558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdc3a9dd1a47415baed4f00867af3273'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.281321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdc3a9dd1a47415baed4f00867af3273'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.281931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdc3a9dd1a47415baed4f00867af3273'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.293379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdc3a9dd1a47415baed4f00867af3273'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.323672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 969/1968] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:38 2022] DELETE /identity/v3/users/fdc3a9dd1a47415baed4f00867af3273 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:39.338226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:39.339993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:39.370107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b6750-2ea5-4d00-8571-db91c5f47ce6 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:39.429553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:39.430983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:39.451862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:39.457177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f2170-9394-47e8-b680-06045f44b7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:39.458685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f2170-9394-47e8-b680-06045f44b7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:39.459487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f2170-9394-47e8-b680-06045f44b7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:39.459636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f2170-9394-47e8-b680-06045f44b7a0 admin admin] PATH_INFO: `/v3/users/688a6a8b487844838831116d873330c3` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:39.462252 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f2170-9394-47e8-b680-06045f44b7a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=688a6a8b487844838831116d873330c3)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:39.466349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f2170-9394-47e8-b680-06045f44b7a0 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:39.522477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:39.534948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd07477f-68c8-426a-8cef-ae623e72666c admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:39.540459 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728db952-fa23-4f51-a07c-297d09caae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:39.541126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728db952-fa23-4f51-a07c-297d09caae06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:39.541417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728db952-fa23-4f51-a07c-297d09caae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:39.541693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728db952-fa23-4f51-a07c-297d09caae06 admin admin] PATH_INFO: `/v3/users/04287d76ff85450a8ea2d27ded8ab6b0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:39.542865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728db952-fa23-4f51-a07c-297d09caae06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04287d76ff85450a8ea2d27ded8ab6b0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:39.544921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728db952-fa23-4f51-a07c-297d09caae06 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:39.690849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f84f2170-9394-47e8-b680-06045f44b7a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '688a6a8b487844838831116d873330c3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.752011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f84f2170-9394-47e8-b680-06045f44b7a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '688a6a8b487844838831116d873330c3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.752701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f84f2170-9394-47e8-b680-06045f44b7a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '688a6a8b487844838831116d873330c3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.753489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f84f2170-9394-47e8-b680-06045f44b7a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '688a6a8b487844838831116d873330c3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.767044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f84f2170-9394-47e8-b680-06045f44b7a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '688a6a8b487844838831116d873330c3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.767685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-728db952-fa23-4f51-a07c-297d09caae06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04287d76ff85450a8ea2d27ded8ab6b0'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.790644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1000/1969] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:39 2022] DELETE /identity/v3/users/688a6a8b487844838831116d873330c3 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:39.795251 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe36738-e7b7-4085-ad22-9866c5a86b2e None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:39.795610 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe36738-e7b7-4085-ad22-9866c5a86b2e None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:39.796014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe36738-e7b7-4085-ad22-9866c5a86b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:39.820151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ffe36738-e7b7-4085-ad22-9866c5a86b2e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:39.821397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ffe36738-e7b7-4085-ad22-9866c5a86b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:39.877225 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-728db952-fa23-4f51-a07c-297d09caae06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04287d76ff85450a8ea2d27ded8ab6b0'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.878176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-728db952-fa23-4f51-a07c-297d09caae06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04287d76ff85450a8ea2d27ded8ab6b0'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.879025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-728db952-fa23-4f51-a07c-297d09caae06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04287d76ff85450a8ea2d27ded8ab6b0'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.887384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-728db952-fa23-4f51-a07c-297d09caae06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04287d76ff85450a8ea2d27ded8ab6b0'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:39.897535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1001/1970] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:23:39.907955 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe36738-e7b7-4085-ad22-9866c5a86b2e None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:39.909516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ffe36738-e7b7-4085-ad22-9866c5a86b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:39.918408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 970/1971] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:39 2022] DELETE /identity/v3/users/04287d76ff85450a8ea2d27ded8ab6b0 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:39.920433 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab434151-7099-4e47-99ca-60d97e016cab None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:39.920915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab434151-7099-4e47-99ca-60d97e016cab None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:39.921524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab434151-7099-4e47-99ca-60d97e016cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:39.946165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ab434151-7099-4e47-99ca-60d97e016cab None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:39.947486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ab434151-7099-4e47-99ca-60d97e016cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:39.977501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe36738-e7b7-4085-ad22-9866c5a86b2e None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:40.012676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 971/1972] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:40.019698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab434151-7099-4e47-99ca-60d97e016cab None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:40.028535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e6382-4155-4ca9-96b0-cb47707501ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:40.029147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e6382-4155-4ca9-96b0-cb47707501ee service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:40.029423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e6382-4155-4ca9-96b0-cb47707501ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:40.029690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e6382-4155-4ca9-96b0-cb47707501ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:40.039041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e6382-4155-4ca9-96b0-cb47707501ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:40.040484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e6382-4155-4ca9-96b0-cb47707501ee service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:40.050299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1002/1973] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:23:40.057941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:40.065088 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e6382-4155-4ca9-96b0-cb47707501ee service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:40.066691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c20e6382-4155-4ca9-96b0-cb47707501ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:40.080428 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e6382-4155-4ca9-96b0-cb47707501ee service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:40.086316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:40.087417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:40.087417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:40.087822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] PATH_INFO: `/v3/projects/f226c630a3a145909ab532f40d0543b7` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:40.090603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f226c630a3a145909ab532f40d0543b7)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:40.091077 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:40.095893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab434151-7099-4e47-99ca-60d97e016cab None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:40.102027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c123df-0d03-49e3-95ef-4587e05e708c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:40.102979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c123df-0d03-49e3-95ef-4587e05e708c service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:40.103114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c123df-0d03-49e3-95ef-4587e05e708c service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:40.103549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c123df-0d03-49e3-95ef-4587e05e708c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:40.115257 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c123df-0d03-49e3-95ef-4587e05e708c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:40.118810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c123df-0d03-49e3-95ef-4587e05e708c service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:40.128043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 972/1974] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:23:40.270337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f226c630a3a145909ab532f40d0543b7', 'request_id': 'req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:40.275710 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f226c630a3a145909ab532f40d0543b7', 'request_id': 'req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:40.370287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f226c630a3a145909ab532f40d0543b7', 'request_id': 'req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:40.396630 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] The token cache is being invalidate because project f226c630a3a145909ab532f40d0543b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:40.397104 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:40.398159 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-18d17f24-5df9-4ae9-b7d8-dc6b954271a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:40.400257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1003/1975] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:40 2022] DELETE /identity/v3/projects/f226c630a3a145909ab532f40d0543b7 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:40.404858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252a00c-99f0-49a2-9c8a-9e7dead78640 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:40.405139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252a00c-99f0-49a2-9c8a-9e7dead78640 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:40.405441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252a00c-99f0-49a2-9c8a-9e7dead78640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:40.426777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-a252a00c-99f0-49a2-9c8a-9e7dead78640 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:40.428340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a252a00c-99f0-49a2-9c8a-9e7dead78640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:40.443996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:40.515123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 973/1976] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:40.525644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b734a51-26d8-43bb-bd92-403a12c90660 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:40.525746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b734a51-26d8-43bb-bd92-403a12c90660 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:40.526025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b734a51-26d8-43bb-bd92-403a12c90660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:40.547051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6b734a51-26d8-43bb-bd92-403a12c90660 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:40.548316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6b734a51-26d8-43bb-bd92-403a12c90660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:40.556503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1004/1977] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:40.560764 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252a00c-99f0-49a2-9c8a-9e7dead78640 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:40.570845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252a00c-99f0-49a2-9c8a-9e7dead78640 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:40.575553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14772299-6e02-487f-96ae-9ad6e71b4faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:40.576203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772299-6e02-487f-96ae-9ad6e71b4faa admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:40.576486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772299-6e02-487f-96ae-9ad6e71b4faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:40.576761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14772299-6e02-487f-96ae-9ad6e71b4faa admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:40.577415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14772299-6e02-487f-96ae-9ad6e71b4faa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:23:40.577725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14772299-6e02-487f-96ae-9ad6e71b4faa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:40.579423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14772299-6e02-487f-96ae-9ad6e71b4faa admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:40.589410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b734a51-26d8-43bb-bd92-403a12c90660 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:40.591242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6b734a51-26d8-43bb-bd92-403a12c90660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:40.598389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 974/1978] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:23:40.602938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14772299-6e02-487f-96ae-9ad6e71b4faa admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:40.605897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b734a51-26d8-43bb-bd92-403a12c90660 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:40.610858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:40.611502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:40.611800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:40.612105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] PATH_INFO: `/v3/projects/fcd02cd0e99d4f7a858200a80f763b1f` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:40.613303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd02cd0e99d4f7a858200a80f763b1f)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:40.615175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:40.615870 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14772299-6e02-487f-96ae-9ad6e71b4faa admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:40.629004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f451de5-c4c0-406d-b1c1-3b8f7e750a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:40.629778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f451de5-c4c0-406d-b1c1-3b8f7e750a1b admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:40.630075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f451de5-c4c0-406d-b1c1-3b8f7e750a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:40.630408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f451de5-c4c0-406d-b1c1-3b8f7e750a1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:40.631785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f451de5-c4c0-406d-b1c1-3b8f7e750a1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:40.633823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f451de5-c4c0-406d-b1c1-3b8f7e750a1b admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:40.772118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 975/1979] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:40 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:40.773444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f451de5-c4c0-406d-b1c1-3b8f7e750a1b admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:40.775018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9f451de5-c4c0-406d-b1c1-3b8f7e750a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:40.785989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd02cd0e99d4f7a858200a80f763b1f', 'request_id': 'req-8bf32e45-cc4b-4623-8645-75e0190049fe'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:40.792392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd02cd0e99d4f7a858200a80f763b1f', 'request_id': 'req-8bf32e45-cc4b-4623-8645-75e0190049fe'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:40.845442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f451de5-c4c0-406d-b1c1-3b8f7e750a1b admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:40.856804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:40.857473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:40.857804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:40.858089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] PATH_INFO: `/v3/projects/b1697b1ad2a64de38ec7edcd2f3e4a6d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:40.860290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1697b1ad2a64de38ec7edcd2f3e4a6d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:40.862334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:40.873389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd02cd0e99d4f7a858200a80f763b1f', 'request_id': 'req-8bf32e45-cc4b-4623-8645-75e0190049fe'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:40.886474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] The token cache is being invalidate because project fcd02cd0e99d4f7a858200a80f763b1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:40.886902 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:40.887861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:40.889755 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1005/1980] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:40 2022] DELETE /identity/v3/projects/fcd02cd0e99d4f7a858200a80f763b1f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:40.890644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:40.893302 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:40.956073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf32e45-cc4b-4623-8645-75e0190049fe admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:40.960810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34941c3d-8db6-4ce6-a41f-46085fa8a3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:40.961476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34941c3d-8db6-4ce6-a41f-46085fa8a3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:40.961753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34941c3d-8db6-4ce6-a41f-46085fa8a3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:40.962030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34941c3d-8db6-4ce6-a41f-46085fa8a3d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:40.967888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34941c3d-8db6-4ce6-a41f-46085fa8a3d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:40.967888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34941c3d-8db6-4ce6-a41f-46085fa8a3d0 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:41.004250 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1697b1ad2a64de38ec7edcd2f3e4a6d', 'request_id': 'req-4ffef918-c04e-43d7-80d1-1880d8a53964'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.103325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1697b1ad2a64de38ec7edcd2f3e4a6d', 'request_id': 'req-4ffef918-c04e-43d7-80d1-1880d8a53964'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.105759 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:41.111273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1006/1981] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:40 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:41.111395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1697b1ad2a64de38ec7edcd2f3e4a6d', 'request_id': 'req-4ffef918-c04e-43d7-80d1-1880d8a53964'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.112195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34941c3d-8db6-4ce6-a41f-46085fa8a3d0 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:41.114527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-34941c3d-8db6-4ce6-a41f-46085fa8a3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:41.122474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] The token cache is being invalidate because project b1697b1ad2a64de38ec7edcd2f3e4a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:41.122760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.123412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.124935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 976/1982] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:40 2022] DELETE /identity/v3/projects/b1697b1ad2a64de38ec7edcd2f3e4a6d => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:41.125682 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:41.126942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:41.127323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34941c3d-8db6-4ce6-a41f-46085fa8a3d0 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:41.185795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:41.186436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:41.186726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:41.187042 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] PATH_INFO: `/v3/users/f3dba5b892a043c599e5e390327fa0f9` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:41.188596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3dba5b892a043c599e5e390327fa0f9)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:41.190567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:41.192788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffef918-c04e-43d7-80d1-1880d8a53964 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:41.197851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12cf90-739c-451c-ac8f-0fa413bea794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:41.198503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12cf90-739c-451c-ac8f-0fa413bea794 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:41.198784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12cf90-739c-451c-ac8f-0fa413bea794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:41.199056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d12cf90-739c-451c-ac8f-0fa413bea794 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:41.199693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12cf90-739c-451c-ac8f-0fa413bea794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:41.201589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d12cf90-739c-451c-ac8f-0fa413bea794 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:41.213137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 977/1983] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:41.214479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12cf90-739c-451c-ac8f-0fa413bea794 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:41.216216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1d12cf90-739c-451c-ac8f-0fa413bea794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:41.234378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d12cf90-739c-451c-ac8f-0fa413bea794 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:41.239807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:41.240450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:41.240728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:41.241052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] PATH_INFO: `/v3/users/677da02f76e94604ba337004eeac53e9` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:41.242325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=677da02f76e94604ba337004eeac53e9)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:41.244350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:41.380785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3dba5b892a043c599e5e390327fa0f9'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.410058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '677da02f76e94604ba337004eeac53e9'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.482767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3dba5b892a043c599e5e390327fa0f9'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.483410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3dba5b892a043c599e5e390327fa0f9'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.484071 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3dba5b892a043c599e5e390327fa0f9'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.492334 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3dba5b892a043c599e5e390327fa0f9'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.511073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1007/1984] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:41 2022] DELETE /identity/v3/users/f3dba5b892a043c599e5e390327fa0f9 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:41.512112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:41.522567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:41.714558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '677da02f76e94604ba337004eeac53e9'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.714558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '677da02f76e94604ba337004eeac53e9'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.714558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '677da02f76e94604ba337004eeac53e9'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.714558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:41.714558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '677da02f76e94604ba337004eeac53e9'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:41.714558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 978/1985] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:41 2022] DELETE /identity/v3/users/677da02f76e94604ba337004eeac53e9 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:41.716873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909cd29f-a3a2-4b4d-91e7-51b05468d210 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:41.716873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c355e01-d23f-4261-8cf4-17e1f6dfbb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:41.716873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c355e01-d23f-4261-8cf4-17e1f6dfbb59 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:41.716873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c355e01-d23f-4261-8cf4-17e1f6dfbb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:41.717254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c355e01-d23f-4261-8cf4-17e1f6dfbb59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:41.717254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c355e01-d23f-4261-8cf4-17e1f6dfbb59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:41.717254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c355e01-d23f-4261-8cf4-17e1f6dfbb59 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:41.717254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1008/1986] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:41.717541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:41.718872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:41.733220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4c685-c70d-4b52-a74d-f76991be4585 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:41.737823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb03e2b-eed9-48a2-968b-c912a90fdd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:41.738505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb03e2b-eed9-48a2-968b-c912a90fdd2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:41.738783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb03e2b-eed9-48a2-968b-c912a90fdd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:41.739060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb03e2b-eed9-48a2-968b-c912a90fdd2b admin admin] PATH_INFO: `/v3/projects/8235aa8e9b0f46e9837bae37e07afb68/users/aab79224668f4beeb57a4314fdadfb46/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:41.754646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb03e2b-eed9-48a2-968b-c912a90fdd2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8235aa8e9b0f46e9837bae37e07afb68, user_id=aab79224668f4beeb57a4314fdadfb46, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:41.757324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb03e2b-eed9-48a2-968b-c912a90fdd2b admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:41.862246 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 979/1987] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:41 2022] PUT /identity/v3/projects/8235aa8e9b0f46e9837bae37e07afb68/users/aab79224668f4beeb57a4314fdadfb46/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:41.877567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c075ff0-d18b-42d9-a1d3-7d6eae8eb910 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:41.878014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c075ff0-d18b-42d9-a1d3-7d6eae8eb910 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:41.878114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c075ff0-d18b-42d9-a1d3-7d6eae8eb910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:41.910374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-7c075ff0-d18b-42d9-a1d3-7d6eae8eb910 None None] MFA Rules not processed for user `aab79224668f4beeb57a4314fdadfb46`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:41.912286 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7c075ff0-d18b-42d9-a1d3-7d6eae8eb910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:42.003371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1009/1988] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:42.021258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb03e2b-eed9-48a2-968b-c912a90fdd2b admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:42.023138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9eb03e2b-eed9-48a2-968b-c912a90fdd2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:42.084365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb03e2b-eed9-48a2-968b-c912a90fdd2b admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:42.088107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e18e82-55a7-41db-bdf0-8b141debe7f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:42.088714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e18e82-55a7-41db-bdf0-8b141debe7f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:42.088998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e18e82-55a7-41db-bdf0-8b141debe7f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:42.089345 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e18e82-55a7-41db-bdf0-8b141debe7f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:42.099753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e18e82-55a7-41db-bdf0-8b141debe7f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:42.102117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e18e82-55a7-41db-bdf0-8b141debe7f6 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:42.109504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c075ff0-d18b-42d9-a1d3-7d6eae8eb910 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:42.110983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7c075ff0-d18b-42d9-a1d3-7d6eae8eb910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:42.115902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 980/1989] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:23:42.144914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e18e82-55a7-41db-bdf0-8b141debe7f6 service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:42.146625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-76e18e82-55a7-41db-bdf0-8b141debe7f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:42.209601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c075ff0-d18b-42d9-a1d3-7d6eae8eb910 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:42.222550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:42.223562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:42.224076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:42.225241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] PATH_INFO: `/v3/projects/4f75e7b1560a4f48ade6f486fbcad9ba` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:42.226910 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f75e7b1560a4f48ade6f486fbcad9ba)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:42.235778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:42.270767 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e18e82-55a7-41db-bdf0-8b141debe7f6 service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:42.277300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:42.278029 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:42.278318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:42.278594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] PATH_INFO: `/v3/projects/696aedebdc3b49b890ce0c62a85ee028` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:42.279731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=696aedebdc3b49b890ce0c62a85ee028)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:42.281779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:42.386146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f75e7b1560a4f48ade6f486fbcad9ba', 'request_id': 'req-edf7f293-f5a1-48d1-ada2-279ffd4f273c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.392025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f75e7b1560a4f48ade6f486fbcad9ba', 'request_id': 'req-edf7f293-f5a1-48d1-ada2-279ffd4f273c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.437537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696aedebdc3b49b890ce0c62a85ee028', 'request_id': 'req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.448212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f75e7b1560a4f48ade6f486fbcad9ba', 'request_id': 'req-edf7f293-f5a1-48d1-ada2-279ffd4f273c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.460076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] The token cache is being invalidate because project 4f75e7b1560a4f48ade6f486fbcad9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:42.460363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.460986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-edf7f293-f5a1-48d1-ada2-279ffd4f273c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.462468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1010/1990] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:42 2022] DELETE /identity/v3/projects/4f75e7b1560a4f48ade6f486fbcad9ba => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:42.467197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701f5c5-8ed4-47fc-b2f4-f76f415a1db8 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:42.467482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701f5c5-8ed4-47fc-b2f4-f76f415a1db8 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:42.467783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701f5c5-8ed4-47fc-b2f4-f76f415a1db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:42.487000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-a701f5c5-8ed4-47fc-b2f4-f76f415a1db8 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:42.488173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a701f5c5-8ed4-47fc-b2f4-f76f415a1db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:42.494758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:42.523149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696aedebdc3b49b890ce0c62a85ee028', 'request_id': 'req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.532103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696aedebdc3b49b890ce0c62a85ee028', 'request_id': 'req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.558118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] The token cache is being invalidate because project 696aedebdc3b49b890ce0c62a85ee028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:42.558499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.559413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-828915d2-6ded-4a58-a6fc-47fbeb5ab3cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.559680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1011/1991] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:23:42.563668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 981/1992] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:42 2022] DELETE /identity/v3/projects/696aedebdc3b49b890ce0c62a85ee028 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:42.565253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701f5c5-8ed4-47fc-b2f4-f76f415a1db8 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:42.566731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a701f5c5-8ed4-47fc-b2f4-f76f415a1db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:42.567349 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9019fe-e246-409a-a263-449fa45c8767 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:42.567640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9019fe-e246-409a-a263-449fa45c8767 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:42.568005 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9019fe-e246-409a-a263-449fa45c8767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:42.578925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701f5c5-8ed4-47fc-b2f4-f76f415a1db8 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:42.584444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:42.585069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:42.585363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:42.585615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] PATH_INFO: `/v3/users/17c3e0ac3f014897bce65fdac4200ef1` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:42.586731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17c3e0ac3f014897bce65fdac4200ef1)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:42.588805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:42.592287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-5a9019fe-e246-409a-a263-449fa45c8767 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:42.592287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-5a9019fe-e246-409a-a263-449fa45c8767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:42.598654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 982/1993] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:42.602722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9019fe-e246-409a-a263-449fa45c8767 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:42.611549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9019fe-e246-409a-a263-449fa45c8767 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:42.616556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73335714-e16c-45c3-bfa4-0c03245ab3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:42.617164 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73335714-e16c-45c3-bfa4-0c03245ab3ef admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:42.617448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73335714-e16c-45c3-bfa4-0c03245ab3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:42.617723 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73335714-e16c-45c3-bfa4-0c03245ab3ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:42.618399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73335714-e16c-45c3-bfa4-0c03245ab3ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:23:42.618760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73335714-e16c-45c3-bfa4-0c03245ab3ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:42.620386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73335714-e16c-45c3-bfa4-0c03245ab3ef admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:42.631906 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 983/1994] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:23:42.636107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73335714-e16c-45c3-bfa4-0c03245ab3ef admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:42.645996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73335714-e16c-45c3-bfa4-0c03245ab3ef admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:42.655511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0dd43-c9e2-4b13-bd3d-87f40c283b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:42.655511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0dd43-c9e2-4b13-bd3d-87f40c283b9d admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:42.655511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0dd43-c9e2-4b13-bd3d-87f40c283b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:42.655511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0dd43-c9e2-4b13-bd3d-87f40c283b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:42.655940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0dd43-c9e2-4b13-bd3d-87f40c283b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:42.655940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0dd43-c9e2-4b13-bd3d-87f40c283b9d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:42.758704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 984/1995] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:42 2022] POST /identity/v3/projects => generated 399 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:42.760302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0dd43-c9e2-4b13-bd3d-87f40c283b9d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:42.795939 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17c3e0ac3f014897bce65fdac4200ef1'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.829746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0dd43-c9e2-4b13-bd3d-87f40c283b9d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:42.834940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab0055-15b1-470c-8a6d-33c24251a48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:42.835631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab0055-15b1-470c-8a6d-33c24251a48d admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:42.835953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab0055-15b1-470c-8a6d-33c24251a48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:42.836226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab0055-15b1-470c-8a6d-33c24251a48d admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:42.836858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab0055-15b1-470c-8a6d-33c24251a48d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:42.838788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab0055-15b1-470c-8a6d-33c24251a48d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:42.900116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17c3e0ac3f014897bce65fdac4200ef1'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.900737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17c3e0ac3f014897bce65fdac4200ef1'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.901346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17c3e0ac3f014897bce65fdac4200ef1'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.909697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17c3e0ac3f014897bce65fdac4200ef1'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:42.928080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1012/1996] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:23:42 2022] DELETE /identity/v3/users/17c3e0ac3f014897bce65fdac4200ef1 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:43.005718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 985/1997] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:42 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:43.009744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:43.011099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:43.078878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6db6fb-ab93-48b9-913d-89b06d96f367 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:43.083508 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17081f-fd66-430c-846e-0ad0fa2230cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:43.084258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17081f-fd66-430c-846e-0ad0fa2230cc admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.084629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17081f-fd66-430c-846e-0ad0fa2230cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.084908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17081f-fd66-430c-846e-0ad0fa2230cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.085569 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17081f-fd66-430c-846e-0ad0fa2230cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:43.087163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17081f-fd66-430c-846e-0ad0fa2230cc admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:43.098125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1013/1998] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:43.102246 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab0055-15b1-470c-8a6d-33c24251a48d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:43.112430 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab0055-15b1-470c-8a6d-33c24251a48d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:43.117519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1298ed-d344-4c45-998c-9b9d8220b65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:43.118125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1298ed-d344-4c45-998c-9b9d8220b65e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.118406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1298ed-d344-4c45-998c-9b9d8220b65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.118681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1298ed-d344-4c45-998c-9b9d8220b65e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.119328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1298ed-d344-4c45-998c-9b9d8220b65e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:43.120958 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1298ed-d344-4c45-998c-9b9d8220b65e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:43.131624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 986/1999] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:23:43.136070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17081f-fd66-430c-846e-0ad0fa2230cc admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:43.151942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17081f-fd66-430c-846e-0ad0fa2230cc admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:43.154666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039107f2-f50f-45c1-804e-a8b90d44be63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:43.155547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039107f2-f50f-45c1-804e-a8b90d44be63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.156007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039107f2-f50f-45c1-804e-a8b90d44be63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.156349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039107f2-f50f-45c1-804e-a8b90d44be63 admin admin] PATH_INFO: `/v3/projects/4497b55c899f48ef81f2dcc670a68f39/users/8f3d64a637fe4b63ada55e073e5d7310/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.176024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039107f2-f50f-45c1-804e-a8b90d44be63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4497b55c899f48ef81f2dcc670a68f39, user_id=8f3d64a637fe4b63ada55e073e5d7310, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:43.179866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039107f2-f50f-45c1-804e-a8b90d44be63 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:43.251948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1014/2000] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:43 2022] PUT /identity/v3/projects/4497b55c899f48ef81f2dcc670a68f39/users/8f3d64a637fe4b63ada55e073e5d7310/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:43.257868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:43.262965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4c30f-1dec-4430-8f0d-1375b8814d9f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.263186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4c30f-1dec-4430-8f0d-1375b8814d9f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.263466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4c30f-1dec-4430-8f0d-1375b8814d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.290465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-95f4c30f-1dec-4430-8f0d-1375b8814d9f None None] MFA Rules not processed for user `8f3d64a637fe4b63ada55e073e5d7310`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:43.291663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-95f4c30f-1dec-4430-8f0d-1375b8814d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:43.354482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 987/2001] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:43 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:43.363787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039107f2-f50f-45c1-804e-a8b90d44be63 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:43.369927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4c30f-1dec-4430-8f0d-1375b8814d9f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:43.371553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-95f4c30f-1dec-4430-8f0d-1375b8814d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:43.456665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4c30f-1dec-4430-8f0d-1375b8814d9f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:43.461639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:43.462279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.462560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.462837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] PATH_INFO: `/v3/projects/c386e8cf1fd6429f9eb70ace7baa1c3a` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.464155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c386e8cf1fd6429f9eb70ace7baa1c3a)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:43.466496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:43.475846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039107f2-f50f-45c1-804e-a8b90d44be63 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:43.482232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41b3ec-b8e3-4787-a23f-d9b25a190e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:43.482931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41b3ec-b8e3-4787-a23f-d9b25a190e40 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.483275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41b3ec-b8e3-4787-a23f-d9b25a190e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.483630 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41b3ec-b8e3-4787-a23f-d9b25a190e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.484409 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41b3ec-b8e3-4787-a23f-d9b25a190e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:43.486378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41b3ec-b8e3-4787-a23f-d9b25a190e40 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:43.571457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c386e8cf1fd6429f9eb70ace7baa1c3a', 'request_id': 'req-6d07bee4-43c2-459b-9da4-261f314bc125'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:43.587567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1015/2002] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:43 2022] POST /identity/v3/projects => generated 399 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:23:43.591469 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41b3ec-b8e3-4787-a23f-d9b25a190e40 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:43.650734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41b3ec-b8e3-4787-a23f-d9b25a190e40 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:43.655503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307133-7539-4b8d-962f-2dd9573618d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:43.656131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307133-7539-4b8d-962f-2dd9573618d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.656412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307133-7539-4b8d-962f-2dd9573618d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.656686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307133-7539-4b8d-962f-2dd9573618d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.657309 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b307133-7539-4b8d-962f-2dd9573618d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:43.659324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b307133-7539-4b8d-962f-2dd9573618d9 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:43.666260 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c386e8cf1fd6429f9eb70ace7baa1c3a', 'request_id': 'req-6d07bee4-43c2-459b-9da4-261f314bc125'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:43.673510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c386e8cf1fd6429f9eb70ace7baa1c3a', 'request_id': 'req-6d07bee4-43c2-459b-9da4-261f314bc125'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:43.684330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] The token cache is being invalidate because project c386e8cf1fd6429f9eb70ace7baa1c3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:23:43.684617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:43.685277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6d07bee4-43c2-459b-9da4-261f314bc125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:23:43.686888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 988/2003] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:23:43 2022] DELETE /identity/v3/projects/c386e8cf1fd6429f9eb70ace7baa1c3a => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:43.691526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622790b7-3f4d-4d5f-bcee-af28b0b312eb None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.691845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622790b7-3f4d-4d5f-bcee-af28b0b312eb None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.692123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622790b7-3f4d-4d5f-bcee-af28b0b312eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.711327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-622790b7-3f4d-4d5f-bcee-af28b0b312eb None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:43.712564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-622790b7-3f4d-4d5f-bcee-af28b0b312eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:43.720469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 989/2004] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:23:43.728367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ada95-d46a-455d-82bf-14ac2da7c34b None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.732315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ada95-d46a-455d-82bf-14ac2da7c34b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.732315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ada95-d46a-455d-82bf-14ac2da7c34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.748343 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ec3ada95-d46a-455d-82bf-14ac2da7c34b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:43.749554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ec3ada95-d46a-455d-82bf-14ac2da7c34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:43.758094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 990/2005] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:23:43.762295 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ada95-d46a-455d-82bf-14ac2da7c34b None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:43.769529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1016/2006] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:43 2022] POST /identity/v3/users => generated 306 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:43.771946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ada95-d46a-455d-82bf-14ac2da7c34b None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:43.773419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307133-7539-4b8d-962f-2dd9573618d9 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:43.775025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2b307133-7539-4b8d-962f-2dd9573618d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:43.776792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f32e6-1b3f-4933-9f11-a6fa764a52ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:43.777384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f32e6-1b3f-4933-9f11-a6fa764a52ec admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.777659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f32e6-1b3f-4933-9f11-a6fa764a52ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.777929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20f32e6-1b3f-4933-9f11-a6fa764a52ec admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.778808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f32e6-1b3f-4933-9f11-a6fa764a52ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:23:43.778979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f32e6-1b3f-4933-9f11-a6fa764a52ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:43.780592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20f32e6-1b3f-4933-9f11-a6fa764a52ec admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:43.787574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307133-7539-4b8d-962f-2dd9573618d9 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:43.792376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ec62c-67cd-4204-b484-230de80462d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:43.792972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ec62c-67cd-4204-b484-230de80462d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.793081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 991/2007] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:23:43.793269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ec62c-67cd-4204-b484-230de80462d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.793524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ec62c-67cd-4204-b484-230de80462d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.794165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ec62c-67cd-4204-b484-230de80462d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:43.795849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ec62c-67cd-4204-b484-230de80462d6 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:43.797726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f32e6-1b3f-4933-9f11-a6fa764a52ec admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:43.807403 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20f32e6-1b3f-4933-9f11-a6fa764a52ec admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:43.808266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1017/2008] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:43.811655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ec62c-67cd-4204-b484-230de80462d6 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:43.812081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532c996-74da-447f-964a-e16fae5b1d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:43.812640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532c996-74da-447f-964a-e16fae5b1d62 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.812926 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532c996-74da-447f-964a-e16fae5b1d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.813256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532c996-74da-447f-964a-e16fae5b1d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.813830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532c996-74da-447f-964a-e16fae5b1d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:43.815712 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532c996-74da-447f-964a-e16fae5b1d62 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:43.821649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ec62c-67cd-4204-b484-230de80462d6 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:43.826259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764d598-55f0-4b47-9767-6eb51ceae279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:43.826900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764d598-55f0-4b47-9767-6eb51ceae279 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:43.827166 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764d598-55f0-4b47-9767-6eb51ceae279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:43.827437 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1764d598-55f0-4b47-9767-6eb51ceae279 admin admin] PATH_INFO: `/v3/projects/232561d0c9e844b0868b0d3b4e15ca78/users/6b9cc1ec67b84a2e921937d4b9fe4a02/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:43.842240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764d598-55f0-4b47-9767-6eb51ceae279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=232561d0c9e844b0868b0d3b4e15ca78, user_id=6b9cc1ec67b84a2e921937d4b9fe4a02, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:43.844917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1764d598-55f0-4b47-9767-6eb51ceae279 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:43.947451 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 992/2009] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:23:43 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:43.951628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532c996-74da-447f-964a-e16fae5b1d62 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:43.975703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1018/2010] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:43 2022] PUT /identity/v3/projects/232561d0c9e844b0868b0d3b4e15ca78/users/6b9cc1ec67b84a2e921937d4b9fe4a02/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:23:43.981689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764d598-55f0-4b47-9767-6eb51ceae279 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:44.015035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532c996-74da-447f-964a-e16fae5b1d62 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:44.052477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1764d598-55f0-4b47-9767-6eb51ceae279 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:44.054425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709e454-c427-46fa-959a-2fd72fdb19f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:44.055060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e454-c427-46fa-959a-2fd72fdb19f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:44.055495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e454-c427-46fa-959a-2fd72fdb19f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:44.055795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e454-c427-46fa-959a-2fd72fdb19f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:44.056454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709e454-c427-46fa-959a-2fd72fdb19f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:44.058161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709e454-c427-46fa-959a-2fd72fdb19f7 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:44.068186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1019/2011] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:44.070756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c8b2-eb09-4c58-9784-19e2f0b2ae3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:44.071375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c8b2-eb09-4c58-9784-19e2f0b2ae3c admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:44.071573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:44.071707 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c8b2-eb09-4c58-9784-19e2f0b2ae3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:44.071955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c8b2-eb09-4c58-9784-19e2f0b2ae3c admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:44.072590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706c8b2-eb09-4c58-9784-19e2f0b2ae3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:44.072712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709e454-c427-46fa-959a-2fd72fdb19f7 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:44.074498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706c8b2-eb09-4c58-9784-19e2f0b2ae3c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:44.082774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709e454-c427-46fa-959a-2fd72fdb19f7 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:44.087480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13945d47-dad2-490c-96bc-135e9b6b3538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:44.088165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13945d47-dad2-490c-96bc-135e9b6b3538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:44.088444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13945d47-dad2-490c-96bc-135e9b6b3538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:44.088721 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13945d47-dad2-490c-96bc-135e9b6b3538 admin admin] PATH_INFO: `/v3/projects/232561d0c9e844b0868b0d3b4e15ca78/users/6b9cc1ec67b84a2e921937d4b9fe4a02/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:44.090629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13945d47-dad2-490c-96bc-135e9b6b3538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=232561d0c9e844b0868b0d3b4e15ca78, user_id=6b9cc1ec67b84a2e921937d4b9fe4a02, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:44.093323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13945d47-dad2-490c-96bc-135e9b6b3538 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:44.103743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1020/2012] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:44 2022] PUT /identity/v3/projects/232561d0c9e844b0868b0d3b4e15ca78/users/6b9cc1ec67b84a2e921937d4b9fe4a02/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:44.114553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72502e1d-0562-4125-b377-6f55df4154ce None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:44.114839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72502e1d-0562-4125-b377-6f55df4154ce None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:44.115116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72502e1d-0562-4125-b377-6f55df4154ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:44.142085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-72502e1d-0562-4125-b377-6f55df4154ce None None] MFA Rules not processed for user `6b9cc1ec67b84a2e921937d4b9fe4a02`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:44.143282 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-72502e1d-0562-4125-b377-6f55df4154ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:44.159973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 993/2013] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:23:43 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:23:44.160996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c8b2-eb09-4c58-9784-19e2f0b2ae3c admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:44.170491 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c8b2-eb09-4c58-9784-19e2f0b2ae3c admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:44.212348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1021/2014] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:44 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:23:44.223095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72502e1d-0562-4125-b377-6f55df4154ce None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:44.224530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-72502e1d-0562-4125-b377-6f55df4154ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:44.228350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de17136-a987-4f05-9bb5-76fe1355a209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:44.229285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17136-a987-4f05-9bb5-76fe1355a209 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:44.229285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17136-a987-4f05-9bb5-76fe1355a209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:44.229719 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de17136-a987-4f05-9bb5-76fe1355a209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:44.230385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de17136-a987-4f05-9bb5-76fe1355a209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:44.231993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de17136-a987-4f05-9bb5-76fe1355a209 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:44.242340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 994/2015] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:44.246247 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de17136-a987-4f05-9bb5-76fe1355a209 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:44.256461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de17136-a987-4f05-9bb5-76fe1355a209 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:44.261097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f6ae7-6fa6-4b84-be8c-a667a101499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:44.261674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6ae7-6fa6-4b84-be8c-a667a101499a admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:44.263743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6ae7-6fa6-4b84-be8c-a667a101499a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:44.263743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6ae7-6fa6-4b84-be8c-a667a101499a admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:44.263743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f6ae7-6fa6-4b84-be8c-a667a101499a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:44.264462 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f6ae7-6fa6-4b84-be8c-a667a101499a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:44.275567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 995/2016] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:23:44.279985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f6ae7-6fa6-4b84-be8c-a667a101499a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:44.282771 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72502e1d-0562-4125-b377-6f55df4154ce None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:44.284618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:44.288146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0669b-508c-45be-b1dc-93e6d2d065f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:44.288777 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0669b-508c-45be-b1dc-93e6d2d065f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:44.292439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0669b-508c-45be-b1dc-93e6d2d065f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:44.292439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0669b-508c-45be-b1dc-93e6d2d065f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:44.292622 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f6ae7-6fa6-4b84-be8c-a667a101499a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:44.294581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cad5da-5ddf-4602-9ee8-9e42e05319a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:44.295208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cad5da-5ddf-4602-9ee8-9e42e05319a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:44.295489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cad5da-5ddf-4602-9ee8-9e42e05319a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:44.295795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cad5da-5ddf-4602-9ee8-9e42e05319a2 admin admin] PATH_INFO: `/v3/projects/146ccff43bd84cf1bc84b9b862190678/users/4bd826253c6b4b5096bee7ffaac04d18/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:44.298933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0669b-508c-45be-b1dc-93e6d2d065f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:44.300391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0669b-508c-45be-b1dc-93e6d2d065f7 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:44.309887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1022/2017] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:44 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:23:44.310840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cad5da-5ddf-4602-9ee8-9e42e05319a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=146ccff43bd84cf1bc84b9b862190678, user_id=4bd826253c6b4b5096bee7ffaac04d18, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:44.313853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cad5da-5ddf-4602-9ee8-9e42e05319a2 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:44.321199 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:44.425743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 996/2018] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:23:44 2022] PUT /identity/v3/projects/146ccff43bd84cf1bc84b9b862190678/users/4bd826253c6b4b5096bee7ffaac04d18/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:23:44.432292 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d40ab-ebda-430a-8655-603deb636016 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:44.433038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d40ab-ebda-430a-8655-603deb636016 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:44.433703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d40ab-ebda-430a-8655-603deb636016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:44.455883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-842d40ab-ebda-430a-8655-603deb636016 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:44.464727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-842d40ab-ebda-430a-8655-603deb636016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:44.521425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1023/2019] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:44.531628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cad5da-5ddf-4602-9ee8-9e42e05319a2 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:44.533795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f1cad5da-5ddf-4602-9ee8-9e42e05319a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:44.562554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d40ab-ebda-430a-8655-603deb636016 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:44.602181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cad5da-5ddf-4602-9ee8-9e42e05319a2 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:44.606223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4d6b1-b523-4cfa-96bb-e7bf5d85312a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:44.606835 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4d6b1-b523-4cfa-96bb-e7bf5d85312a service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:44.607123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4d6b1-b523-4cfa-96bb-e7bf5d85312a service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:44.607410 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4d6b1-b523-4cfa-96bb-e7bf5d85312a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:44.617688 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4d6b1-b523-4cfa-96bb-e7bf5d85312a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:44.619264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4d6b1-b523-4cfa-96bb-e7bf5d85312a service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:44.627535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d40ab-ebda-430a-8655-603deb636016 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:44.630353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 997/2020] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:23:44.632224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112e42c-6d37-4617-87d5-a892bc5da779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:44.632848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112e42c-6d37-4617-87d5-a892bc5da779 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:44.633131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112e42c-6d37-4617-87d5-a892bc5da779 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:44.633408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112e42c-6d37-4617-87d5-a892bc5da779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:44.635170 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9112e42c-6d37-4617-87d5-a892bc5da779 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:44.698691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112e42c-6d37-4617-87d5-a892bc5da779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:44.700264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112e42c-6d37-4617-87d5-a892bc5da779 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:44.709823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1024/2021] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:44 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:23:44.785949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:44.842898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:45.248901 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:45.723515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:23:45.799004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5067a-e6e7-4da6-9baa-a772a2b13ddd None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:45.799340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5067a-e6e7-4da6-9baa-a772a2b13ddd None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:45.799554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5067a-e6e7-4da6-9baa-a772a2b13ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:45.833160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-82b5067a-e6e7-4da6-9baa-a772a2b13ddd None None] MFA Rules not processed for user `4bd826253c6b4b5096bee7ffaac04d18`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:45.834723 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-82b5067a-e6e7-4da6-9baa-a772a2b13ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:45.913843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 998/2022] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:45.926550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112e42c-6d37-4617-87d5-a892bc5da779 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:23:45.934814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112e42c-6d37-4617-87d5-a892bc5da779 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:23:45.938491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909dca0-fd1d-4ac6-a3cc-0b36c6a831e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:23:45.939022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909dca0-fd1d-4ac6-a3cc-0b36c6a831e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:45.939232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909dca0-fd1d-4ac6-a3cc-0b36c6a831e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:45.939508 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909dca0-fd1d-4ac6-a3cc-0b36c6a831e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:45.948205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909dca0-fd1d-4ac6-a3cc-0b36c6a831e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:23:45.949649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909dca0-fd1d-4ac6-a3cc-0b36c6a831e5 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:23:45.958824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1025/2023] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:23:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:23:55.313153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bcdbf-fb95-43e4-ac99-07a7116606d1 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:23:55.313771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bcdbf-fb95-43e4-ac99-07a7116606d1 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:23:55.313771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bcdbf-fb95-43e4-ac99-07a7116606d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:23:55.333504 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-042bcdbf-fb95-43e4-ac99-07a7116606d1 None None] MFA Rules not processed for user `4bd826253c6b4b5096bee7ffaac04d18`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:23:55.334753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-042bcdbf-fb95-43e4-ac99-07a7116606d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:23:55.341822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 999/2024] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:23:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:23:55.954414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:05.327891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:07.292719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909dca0-fd1d-4ac6-a3cc-0b36c6a831e5 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:07.304921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909dca0-fd1d-4ac6-a3cc-0b36c6a831e5 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:07.309666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:07.310294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:07.310578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:07.310859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] PATH_INFO: `/v3/users/8f3d64a637fe4b63ada55e073e5d7310` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:07.312199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f3d64a637fe4b63ada55e073e5d7310)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:07.314146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:07.516413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f3d64a637fe4b63ada55e073e5d7310'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:07.584550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f3d64a637fe4b63ada55e073e5d7310'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:07.585217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f3d64a637fe4b63ada55e073e5d7310'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:07.585826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f3d64a637fe4b63ada55e073e5d7310'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:07.593284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f3d64a637fe4b63ada55e073e5d7310'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:07.610609 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1026/2025] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:07 2022] DELETE /identity/v3/users/8f3d64a637fe4b63ada55e073e5d7310 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:07.615158 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535a8a2-9873-4078-aaf0-3c3cb302ef54 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:07.615804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535a8a2-9873-4078-aaf0-3c3cb302ef54 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:07.615804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535a8a2-9873-4078-aaf0-3c3cb302ef54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:07.634773 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-5535a8a2-9873-4078-aaf0-3c3cb302ef54 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:07.636000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-5535a8a2-9873-4078-aaf0-3c3cb302ef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:07.695476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1000/2026] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:07.708064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:07.709740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:07.775539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f5567-fd4d-4f79-870c-00fb6d1ce4bf admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:07.779388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f010e-afb4-4c61-866c-b83a2e910441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:07.780245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f010e-afb4-4c61-866c-b83a2e910441 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:07.780484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f010e-afb4-4c61-866c-b83a2e910441 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:07.780762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f010e-afb4-4c61-866c-b83a2e910441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:07.794767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f010e-afb4-4c61-866c-b83a2e910441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:07.796505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024f010e-afb4-4c61-866c-b83a2e910441 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:07.809107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1027/2027] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:08.586219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535a8a2-9873-4078-aaf0-3c3cb302ef54 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:08.588046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-5535a8a2-9873-4078-aaf0-3c3cb302ef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:08.601726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535a8a2-9873-4078-aaf0-3c3cb302ef54 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:08.606636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:08.607411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:08.607810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:08.608222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] PATH_INFO: `/v3/projects/4497b55c899f48ef81f2dcc670a68f39` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:08.609441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4497b55c899f48ef81f2dcc670a68f39)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:08.611654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:08.733395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4497b55c899f48ef81f2dcc670a68f39', 'request_id': 'req-1ba253fc-a6ed-4211-8ce2-860cce4ab892'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:08.821807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4497b55c899f48ef81f2dcc670a68f39', 'request_id': 'req-1ba253fc-a6ed-4211-8ce2-860cce4ab892'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:08.828821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4497b55c899f48ef81f2dcc670a68f39', 'request_id': 'req-1ba253fc-a6ed-4211-8ce2-860cce4ab892'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:08.841524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] The token cache is being invalidate because project 4497b55c899f48ef81f2dcc670a68f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:08.842041 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:08.842846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:08.844566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1001/2028] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:08 2022] DELETE /identity/v3/projects/4497b55c899f48ef81f2dcc670a68f39 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:08.849372 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f010e-afb4-4c61-866c-b83a2e910441 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:08.859080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f010e-afb4-4c61-866c-b83a2e910441 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:08.919618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1e241-e9df-409c-a5d5-0be8e192b64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:08.920965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1e241-e9df-409c-a5d5-0be8e192b64d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:08.921346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1e241-e9df-409c-a5d5-0be8e192b64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:08.921721 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1e241-e9df-409c-a5d5-0be8e192b64d admin admin] PATH_INFO: `/v3/users/6b9cc1ec67b84a2e921937d4b9fe4a02` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:08.923129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1e241-e9df-409c-a5d5-0be8e192b64d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b9cc1ec67b84a2e921937d4b9fe4a02)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:08.925209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1e241-e9df-409c-a5d5-0be8e192b64d admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:08.942932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:09.138466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-94a1e241-e9df-409c-a5d5-0be8e192b64d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b9cc1ec67b84a2e921937d4b9fe4a02'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:09.230289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-94a1e241-e9df-409c-a5d5-0be8e192b64d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b9cc1ec67b84a2e921937d4b9fe4a02'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:09.230914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-94a1e241-e9df-409c-a5d5-0be8e192b64d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b9cc1ec67b84a2e921937d4b9fe4a02'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:09.231526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-94a1e241-e9df-409c-a5d5-0be8e192b64d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b9cc1ec67b84a2e921937d4b9fe4a02'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:09.239967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-94a1e241-e9df-409c-a5d5-0be8e192b64d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b9cc1ec67b84a2e921937d4b9fe4a02'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:09.257942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1028/2029] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:08 2022] DELETE /identity/v3/users/6b9cc1ec67b84a2e921937d4b9fe4a02 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:09.681804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:09.683519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:09.699331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba253fc-a6ed-4211-8ce2-860cce4ab892 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:09.776311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:09.778832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:09.779166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:09.781637 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] PATH_INFO: `/v3/projects/232561d0c9e844b0868b0d3b4e15ca78` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:09.781637 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=232561d0c9e844b0868b0d3b4e15ca78)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:09.783100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:09.946680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:09.965292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '232561d0c9e844b0868b0d3b4e15ca78', 'request_id': 'req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:10.061339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '232561d0c9e844b0868b0d3b4e15ca78', 'request_id': 'req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:10.068266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '232561d0c9e844b0868b0d3b4e15ca78', 'request_id': 'req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:10.082404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] The token cache is being invalidate because project 232561d0c9e844b0868b0d3b4e15ca78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:10.082404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:10.082404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4d530cd9-87ac-4f3a-88ac-f6ae0a0688ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:10.084448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1002/2030] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:09 2022] DELETE /identity/v3/projects/232561d0c9e844b0868b0d3b4e15ca78 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:10.089774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57080a21-9f19-4032-9793-fb1291ab9ba4 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:10.090250 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57080a21-9f19-4032-9793-fb1291ab9ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:10.090521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57080a21-9f19-4032-9793-fb1291ab9ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:10.113041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-57080a21-9f19-4032-9793-fb1291ab9ba4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:10.114273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-57080a21-9f19-4032-9793-fb1291ab9ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:10.175261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1029/2031] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:10.182630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90769f9e-4e9e-4aea-82d7-e3d6b919ae52 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:10.182921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90769f9e-4e9e-4aea-82d7-e3d6b919ae52 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:10.183201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90769f9e-4e9e-4aea-82d7-e3d6b919ae52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:10.203621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-90769f9e-4e9e-4aea-82d7-e3d6b919ae52 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:10.204896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-90769f9e-4e9e-4aea-82d7-e3d6b919ae52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:10.214833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1003/2032] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:10.220178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57080a21-9f19-4032-9793-fb1291ab9ba4 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:10.235806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57080a21-9f19-4032-9793-fb1291ab9ba4 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:10.252685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2169450-d4db-4cb1-b420-df6867343b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:10.253328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2169450-d4db-4cb1-b420-df6867343b30 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:10.253611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2169450-d4db-4cb1-b420-df6867343b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:10.253882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2169450-d4db-4cb1-b420-df6867343b30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:10.254534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2169450-d4db-4cb1-b420-df6867343b30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:10.254833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2169450-d4db-4cb1-b420-df6867343b30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:10.256484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2169450-d4db-4cb1-b420-df6867343b30 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:10.270220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1030/2033] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:24:10.274389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90769f9e-4e9e-4aea-82d7-e3d6b919ae52 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:10.284250 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90769f9e-4e9e-4aea-82d7-e3d6b919ae52 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:10.291021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3cb0d-dbed-4fab-b58f-ff404ba9e28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:10.291658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3cb0d-dbed-4fab-b58f-ff404ba9e28d admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:10.291990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3cb0d-dbed-4fab-b58f-ff404ba9e28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:10.292270 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3cb0d-dbed-4fab-b58f-ff404ba9e28d admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:10.292914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3cb0d-dbed-4fab-b58f-ff404ba9e28d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:10.294832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3cb0d-dbed-4fab-b58f-ff404ba9e28d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:10.406715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1004/2034] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:10 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:10.407949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2169450-d4db-4cb1-b420-df6867343b30 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:10.421034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2169450-d4db-4cb1-b420-df6867343b30 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:10.510884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3730f04-1639-421a-8aaa-c951fb0982e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:10.511970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3730f04-1639-421a-8aaa-c951fb0982e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:10.512456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3730f04-1639-421a-8aaa-c951fb0982e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:10.512883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3730f04-1639-421a-8aaa-c951fb0982e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:10.513669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3730f04-1639-421a-8aaa-c951fb0982e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:10.516194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3730f04-1639-421a-8aaa-c951fb0982e3 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:10.616536 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1031/2035] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:10 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:10.620959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3cb0d-dbed-4fab-b58f-ff404ba9e28d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:10.631887 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3cb0d-dbed-4fab-b58f-ff404ba9e28d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:10.637263 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bba8f4-4354-429b-89c6-34a3a00ded3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:10.638449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bba8f4-4354-429b-89c6-34a3a00ded3e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:10.638449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bba8f4-4354-429b-89c6-34a3a00ded3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:10.638711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bba8f4-4354-429b-89c6-34a3a00ded3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:10.639498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bba8f4-4354-429b-89c6-34a3a00ded3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:10.641377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bba8f4-4354-429b-89c6-34a3a00ded3e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:10.657336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1005/2036] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:10.658727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3730f04-1639-421a-8aaa-c951fb0982e3 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:10.669599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3730f04-1639-421a-8aaa-c951fb0982e3 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:10.681816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e20eb9-b673-41bd-8075-43dcec38376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:10.682493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e20eb9-b673-41bd-8075-43dcec38376c admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:10.682778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e20eb9-b673-41bd-8075-43dcec38376c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:10.683074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e20eb9-b673-41bd-8075-43dcec38376c admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:10.684456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e20eb9-b673-41bd-8075-43dcec38376c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:10.686073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e20eb9-b673-41bd-8075-43dcec38376c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:10.692885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:10.702648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1032/2037] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:10.704528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bba8f4-4354-429b-89c6-34a3a00ded3e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:10.715856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bba8f4-4354-429b-89c6-34a3a00ded3e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:10.725036 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0056661-7710-4d13-bb3d-61f9cef5bb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:10.725036 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0056661-7710-4d13-bb3d-61f9cef5bb97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:10.725036 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0056661-7710-4d13-bb3d-61f9cef5bb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:10.725036 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0056661-7710-4d13-bb3d-61f9cef5bb97 admin admin] PATH_INFO: `/v3/projects/61d2531e30904c9cafac2cc80a74cf9b/users/a779af5cc55943dd9a8dfd66c0bf6023/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:10.739334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0056661-7710-4d13-bb3d-61f9cef5bb97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61d2531e30904c9cafac2cc80a74cf9b, user_id=a779af5cc55943dd9a8dfd66c0bf6023, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:10.742020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0056661-7710-4d13-bb3d-61f9cef5bb97 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:10.861193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1006/2038] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:10 2022] PUT /identity/v3/projects/61d2531e30904c9cafac2cc80a74cf9b/users/a779af5cc55943dd9a8dfd66c0bf6023/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:10.872200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81610716-5358-436c-baed-da03b523ec79 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:10.872482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81610716-5358-436c-baed-da03b523ec79 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:10.872756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81610716-5358-436c-baed-da03b523ec79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:10.903106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-81610716-5358-436c-baed-da03b523ec79 None None] MFA Rules not processed for user `a779af5cc55943dd9a8dfd66c0bf6023`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:10.903106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-81610716-5358-436c-baed-da03b523ec79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:10.906269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0056661-7710-4d13-bb3d-61f9cef5bb97 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:10.909085 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a0056661-7710-4d13-bb3d-61f9cef5bb97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:10.979149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1033/2039] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:10.980143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81610716-5358-436c-baed-da03b523ec79 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:10.999439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0056661-7710-4d13-bb3d-61f9cef5bb97 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:11.007617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:11.008678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:11.009144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:11.012012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] PATH_INFO: `/v3/users/2eff82d212164719aacb10143d365259` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:11.013448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eff82d212164719aacb10143d365259)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:11.016436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:11.076672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81610716-5358-436c-baed-da03b523ec79 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:11.081895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afc6d1-f503-4635-9ef1-293854d9eb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:11.082515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afc6d1-f503-4635-9ef1-293854d9eb49 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:11.082810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afc6d1-f503-4635-9ef1-293854d9eb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:11.083084 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afc6d1-f503-4635-9ef1-293854d9eb49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:11.083748 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afc6d1-f503-4635-9ef1-293854d9eb49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:11.085656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afc6d1-f503-4635-9ef1-293854d9eb49 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:11.194687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1034/2040] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:10 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:11.198640 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afc6d1-f503-4635-9ef1-293854d9eb49 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:11.247114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eff82d212164719aacb10143d365259'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:11.251938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:DELETE=14,keystone:SELECT=82,keystone:INSERT=4 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:11.268426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afc6d1-f503-4635-9ef1-293854d9eb49 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:11.272034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3f1af-0b40-49ef-a275-6e7cc219ccde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:11.272639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3f1af-0b40-49ef-a275-6e7cc219ccde admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:11.272917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3f1af-0b40-49ef-a275-6e7cc219ccde admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:11.273188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3f1af-0b40-49ef-a275-6e7cc219ccde admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:11.273817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3f1af-0b40-49ef-a275-6e7cc219ccde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:11.275807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3f1af-0b40-49ef-a275-6e7cc219ccde admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:11.321141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eff82d212164719aacb10143d365259'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:11.321765 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eff82d212164719aacb10143d365259'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:11.322381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eff82d212164719aacb10143d365259'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:11.329697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eff82d212164719aacb10143d365259'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:11.346444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1007/2041] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:10 2022] DELETE /identity/v3/users/2eff82d212164719aacb10143d365259 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:11.390926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1035/2042] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:11 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:11.395053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:11.396488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:11.477003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b710b5-6424-44cc-9cd4-7dec50f976ca admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:11.482066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2292f04-5717-4845-8bff-b598b506cb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:11.483175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2292f04-5717-4845-8bff-b598b506cb4f admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:11.483850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2292f04-5717-4845-8bff-b598b506cb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:11.483850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2292f04-5717-4845-8bff-b598b506cb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:11.487461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2292f04-5717-4845-8bff-b598b506cb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:11.487461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2292f04-5717-4845-8bff-b598b506cb4f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:11.504097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1008/2043] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:11.508329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3f1af-0b40-49ef-a275-6e7cc219ccde admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:11.516453 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:11.525966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3f1af-0b40-49ef-a275-6e7cc219ccde admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:11.531340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35a4b4-dd23-4b88-a6cf-2c268c97a53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:11.532031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35a4b4-dd23-4b88-a6cf-2c268c97a53b admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:11.532315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35a4b4-dd23-4b88-a6cf-2c268c97a53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:11.532592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35a4b4-dd23-4b88-a6cf-2c268c97a53b admin admin] PATH_INFO: `/v3/projects/3b4019c2603543a0a01f956d39a67055/users/9fddccd9808e4e30af0342e5ffb56ac7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:11.548869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35a4b4-dd23-4b88-a6cf-2c268c97a53b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4019c2603543a0a01f956d39a67055, user_id=9fddccd9808e4e30af0342e5ffb56ac7, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:11.551632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35a4b4-dd23-4b88-a6cf-2c268c97a53b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:11.671789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1036/2044] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:11 2022] PUT /identity/v3/projects/3b4019c2603543a0a01f956d39a67055/users/9fddccd9808e4e30af0342e5ffb56ac7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:11.675846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2292f04-5717-4845-8bff-b598b506cb4f admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:11.675999 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35a4b4-dd23-4b88-a6cf-2c268c97a53b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:11.677960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c2292f04-5717-4845-8bff-b598b506cb4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:11.711236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2292f04-5717-4845-8bff-b598b506cb4f admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:11.752601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35a4b4-dd23-4b88-a6cf-2c268c97a53b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:11.757691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137124e6-c8be-403d-8622-13fc3ba0181b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:11.758305 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137124e6-c8be-403d-8622-13fc3ba0181b admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:11.758590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137124e6-c8be-403d-8622-13fc3ba0181b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:11.758868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137124e6-c8be-403d-8622-13fc3ba0181b admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:11.759614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137124e6-c8be-403d-8622-13fc3ba0181b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:11.761336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137124e6-c8be-403d-8622-13fc3ba0181b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:11.779313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1037/2045] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:11.780656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137124e6-c8be-403d-8622-13fc3ba0181b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:11.790111 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:11.790755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:11.790961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137124e6-c8be-403d-8622-13fc3ba0181b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:11.791062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:11.791298 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] PATH_INFO: `/v3/projects/f6d21a3e70ba4789bfdfe42476d03a06` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:11.792434 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6d21a3e70ba4789bfdfe42476d03a06)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:11.794596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:11.796112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654224d6-9673-4f5a-adf8-54a6160dbd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:11.796753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654224d6-9673-4f5a-adf8-54a6160dbd2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:11.797037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654224d6-9673-4f5a-adf8-54a6160dbd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:11.797315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654224d6-9673-4f5a-adf8-54a6160dbd2d admin admin] PATH_INFO: `/v3/projects/3b4019c2603543a0a01f956d39a67055/users/9fddccd9808e4e30af0342e5ffb56ac7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:11.799243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654224d6-9673-4f5a-adf8-54a6160dbd2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b4019c2603543a0a01f956d39a67055, user_id=9fddccd9808e4e30af0342e5ffb56ac7, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:11.801971 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654224d6-9673-4f5a-adf8-54a6160dbd2d admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:11.813499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1038/2046] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:11 2022] PUT /identity/v3/projects/3b4019c2603543a0a01f956d39a67055/users/9fddccd9808e4e30af0342e5ffb56ac7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:11.826752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b31896-0c11-43b9-800c-4e5774404aaa None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:11.827007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b31896-0c11-43b9-800c-4e5774404aaa None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:11.827281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b31896-0c11-43b9-800c-4e5774404aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:11.855548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e8b31896-0c11-43b9-800c-4e5774404aaa None None] MFA Rules not processed for user `9fddccd9808e4e30af0342e5ffb56ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:11.856777 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e8b31896-0c11-43b9-800c-4e5774404aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:11.933699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1039/2047] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:11.934118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b31896-0c11-43b9-800c-4e5774404aaa None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:11.935635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e8b31896-0c11-43b9-800c-4e5774404aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:11.942518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6d21a3e70ba4789bfdfe42476d03a06', 'request_id': 'req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:12.000340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b31896-0c11-43b9-800c-4e5774404aaa None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:12.056444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0353e-a5b4-43d6-89a7-078439a21ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:12.057076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0353e-a5b4-43d6-89a7-078439a21ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.057382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0353e-a5b4-43d6-89a7-078439a21ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.057657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0353e-a5b4-43d6-89a7-078439a21ba2 admin admin] PATH_INFO: `/v3/users/9d617e7c7bdd4bcfb342c7b6ff223bb7` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.058790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0353e-a5b4-43d6-89a7-078439a21ba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d617e7c7bdd4bcfb342c7b6ff223bb7)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:12.060803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0353e-a5b4-43d6-89a7-078439a21ba2 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:12.061471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6d21a3e70ba4789bfdfe42476d03a06', 'request_id': 'req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:12.069067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6d21a3e70ba4789bfdfe42476d03a06', 'request_id': 'req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:12.079967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] The token cache is being invalidate because project f6d21a3e70ba4789bfdfe42476d03a06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:12.080258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:12.080926 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:12.083063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1009/2048] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:11 2022] DELETE /identity/v3/projects/f6d21a3e70ba4789bfdfe42476d03a06 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:12.083873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:12.085277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:12.140743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d054b38a-756e-40ca-b3df-b90d9e0bc7bd admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:12.145696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceef43b-a064-4d98-9fa5-b577fcc4db86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:12.146265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceef43b-a064-4d98-9fa5-b577fcc4db86 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.146547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceef43b-a064-4d98-9fa5-b577fcc4db86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.146820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceef43b-a064-4d98-9fa5-b577fcc4db86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.148906 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0ceef43b-a064-4d98-9fa5-b577fcc4db86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:12.173308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:12.211578 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceef43b-a064-4d98-9fa5-b577fcc4db86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:12.213045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceef43b-a064-4d98-9fa5-b577fcc4db86 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:12.222098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1010/2049] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:12.224638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c23ffc-5429-4a52-8a06-cd372fad60de None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.225256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c23ffc-5429-4a52-8a06-cd372fad60de None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.225644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c23ffc-5429-4a52-8a06-cd372fad60de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.240906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fbd0353e-a5b4-43d6-89a7-078439a21ba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d617e7c7bdd4bcfb342c7b6ff223bb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:12.263188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-37c23ffc-5429-4a52-8a06-cd372fad60de None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:12.264383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-37c23ffc-5429-4a52-8a06-cd372fad60de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:12.277310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1011/2050] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:12.285218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230bb5a-90c1-4bb2-8b9c-530b743ec078 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.285507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230bb5a-90c1-4bb2-8b9c-530b743ec078 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.285830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230bb5a-90c1-4bb2-8b9c-530b743ec078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.319921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fbd0353e-a5b4-43d6-89a7-078439a21ba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d617e7c7bdd4bcfb342c7b6ff223bb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:12.319921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fbd0353e-a5b4-43d6-89a7-078439a21ba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d617e7c7bdd4bcfb342c7b6ff223bb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:12.319921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fbd0353e-a5b4-43d6-89a7-078439a21ba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d617e7c7bdd4bcfb342c7b6ff223bb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:12.320312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-8230bb5a-90c1-4bb2-8b9c-530b743ec078 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:12.322624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8230bb5a-90c1-4bb2-8b9c-530b743ec078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:12.327369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fbd0353e-a5b4-43d6-89a7-078439a21ba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d617e7c7bdd4bcfb342c7b6ff223bb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:12.345860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1040/2051] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:11 2022] DELETE /identity/v3/users/9d617e7c7bdd4bcfb342c7b6ff223bb7 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:12.349999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1012/2052] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:12.353185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd992e94-85d0-4c2b-950f-b345df61c6c9 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.353619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd992e94-85d0-4c2b-950f-b345df61c6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.354035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd992e94-85d0-4c2b-950f-b345df61c6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.354538 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230bb5a-90c1-4bb2-8b9c-530b743ec078 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:12.365347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230bb5a-90c1-4bb2-8b9c-530b743ec078 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:12.375583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-fd992e94-85d0-4c2b-950f-b345df61c6c9 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:12.377026 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fd992e94-85d0-4c2b-950f-b345df61c6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:12.393491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:12.426238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa331a-f99e-4eb3-9837-8972857a6339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:12.426850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa331a-f99e-4eb3-9837-8972857a6339 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.427140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa331a-f99e-4eb3-9837-8972857a6339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.427420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa331a-f99e-4eb3-9837-8972857a6339 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.428099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa331a-f99e-4eb3-9837-8972857a6339 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:12.428408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa331a-f99e-4eb3-9837-8972857a6339 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:12.430011 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa331a-f99e-4eb3-9837-8972857a6339 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:12.439299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1041/2053] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:12.445807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1013/2054] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:24:12.448872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd992e94-85d0-4c2b-950f-b345df61c6c9 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:12.449465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa331a-f99e-4eb3-9837-8972857a6339 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:12.451200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-24aa331a-f99e-4eb3-9837-8972857a6339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:12.460306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd992e94-85d0-4c2b-950f-b345df61c6c9 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:12.464865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51513f0f-c253-490c-aa38-ee5c79f75de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:12.465470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51513f0f-c253-490c-aa38-ee5c79f75de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.466367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51513f0f-c253-490c-aa38-ee5c79f75de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.466604 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51513f0f-c253-490c-aa38-ee5c79f75de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.467243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51513f0f-c253-490c-aa38-ee5c79f75de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:12.469235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51513f0f-c253-490c-aa38-ee5c79f75de3 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:12.510186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa331a-f99e-4eb3-9837-8972857a6339 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:12.513811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3273ca7-464f-455a-8847-7952db2d563b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:12.516717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3273ca7-464f-455a-8847-7952db2d563b service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.516717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3273ca7-464f-455a-8847-7952db2d563b service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.516717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3273ca7-464f-455a-8847-7952db2d563b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.524400 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3273ca7-464f-455a-8847-7952db2d563b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:12.525871 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3273ca7-464f-455a-8847-7952db2d563b service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:12.536950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1014/2055] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:12.547844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1042/2056] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:12 2022] POST /identity/v3/projects => generated 393 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:12.549563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3273ca7-464f-455a-8847-7952db2d563b service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:12.560201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3273ca7-464f-455a-8847-7952db2d563b service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:12.623895 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35075d9-a65d-4099-9f4c-0d7e2448139a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:12.624526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35075d9-a65d-4099-9f4c-0d7e2448139a admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.624785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35075d9-a65d-4099-9f4c-0d7e2448139a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.625058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35075d9-a65d-4099-9f4c-0d7e2448139a admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.625699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35075d9-a65d-4099-9f4c-0d7e2448139a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:12.627608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35075d9-a65d-4099-9f4c-0d7e2448139a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:12.742675 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1015/2057] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:12 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:12.750621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51513f0f-c253-490c-aa38-ee5c79f75de3 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:12.762123 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51513f0f-c253-490c-aa38-ee5c79f75de3 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:12.768065 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ac2c4-d1b1-4045-a826-2c0860cac917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:12.768778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ac2c4-d1b1-4045-a826-2c0860cac917 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.769185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ac2c4-d1b1-4045-a826-2c0860cac917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.769557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ac2c4-d1b1-4045-a826-2c0860cac917 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.770336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ac2c4-d1b1-4045-a826-2c0860cac917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:12.772310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ac2c4-d1b1-4045-a826-2c0860cac917 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:12.782937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1043/2058] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:12.787582 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35075d9-a65d-4099-9f4c-0d7e2448139a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:12.809561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35075d9-a65d-4099-9f4c-0d7e2448139a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:12.815587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc7aff-1fb6-437f-85fa-4999a5bfba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:12.815727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc7aff-1fb6-437f-85fa-4999a5bfba59 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.816016 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc7aff-1fb6-437f-85fa-4999a5bfba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.816294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc7aff-1fb6-437f-85fa-4999a5bfba59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.817090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc7aff-1fb6-437f-85fa-4999a5bfba59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:12.818921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc7aff-1fb6-437f-85fa-4999a5bfba59 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:12.831840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1016/2059] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:12.836426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ac2c4-d1b1-4045-a826-2c0860cac917 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:12.851971 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ac2c4-d1b1-4045-a826-2c0860cac917 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:12.853091 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801d263-1a86-47c4-8e2e-934dd827a20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:12.853777 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801d263-1a86-47c4-8e2e-934dd827a20e admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.854589 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801d263-1a86-47c4-8e2e-934dd827a20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.854754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801d263-1a86-47c4-8e2e-934dd827a20e admin admin] PATH_INFO: `/v3/projects/04611beea2e54fa0be05a5eaf4715323/users/d3da95b627b74c28b921586d0da89521/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:12.872885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801d263-1a86-47c4-8e2e-934dd827a20e admin admin] RBAC: Authorizing `identity:create_grant(project_id=04611beea2e54fa0be05a5eaf4715323, user_id=d3da95b627b74c28b921586d0da89521, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:12.875576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801d263-1a86-47c4-8e2e-934dd827a20e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:12.979105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1044/2060] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:12 2022] PUT /identity/v3/projects/04611beea2e54fa0be05a5eaf4715323/users/d3da95b627b74c28b921586d0da89521/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:12.979249 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76709da-8676-48fb-997d-75b16bd7ad18 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:12.979249 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76709da-8676-48fb-997d-75b16bd7ad18 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:12.979814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76709da-8676-48fb-997d-75b16bd7ad18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:13.016903 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-e76709da-8676-48fb-997d-75b16bd7ad18 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:13.018442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e76709da-8676-48fb-997d-75b16bd7ad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:13.021982 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:13.074220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801d263-1a86-47c4-8e2e-934dd827a20e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:13.076078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0801d263-1a86-47c4-8e2e-934dd827a20e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:13.085110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1017/2061] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:13.095039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801d263-1a86-47c4-8e2e-934dd827a20e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:13.096153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76709da-8676-48fb-997d-75b16bd7ad18 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:13.098550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e76709da-8676-48fb-997d-75b16bd7ad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:13.101757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:13.102402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:13.102678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:13.102957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] PATH_INFO: `/v3/projects/c2201e585106491d9551cd62ec095b45` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:13.104177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2201e585106491d9551cd62ec095b45)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:13.108000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:13.162547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76709da-8676-48fb-997d-75b16bd7ad18 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:13.166255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc12bb4-ea58-4442-97f2-d4b6fbbce912 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:13.166869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc12bb4-ea58-4442-97f2-d4b6fbbce912 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:13.167140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc12bb4-ea58-4442-97f2-d4b6fbbce912 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:13.167411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc12bb4-ea58-4442-97f2-d4b6fbbce912 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:13.176517 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc12bb4-ea58-4442-97f2-d4b6fbbce912 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:13.177937 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc12bb4-ea58-4442-97f2-d4b6fbbce912 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:13.187215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1018/2062] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:24:13.235689 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:13.252804 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2201e585106491d9551cd62ec095b45', 'request_id': 'req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:13.261059 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2201e585106491d9551cd62ec095b45', 'request_id': 'req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:13.369734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2201e585106491d9551cd62ec095b45', 'request_id': 'req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:13.384135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] The token cache is being invalidate because project c2201e585106491d9551cd62ec095b45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:13.384507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:13.385256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:13.386850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1045/2063] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:13 2022] DELETE /identity/v3/projects/c2201e585106491d9551cd62ec095b45 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:13.390879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc12bb4-ea58-4442-97f2-d4b6fbbce912 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:13.401267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc12bb4-ea58-4442-97f2-d4b6fbbce912 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:13.462097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:13.462749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:13.463034 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:13.463315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] PATH_INFO: `/v3/users/b032ba8bfcc54f1d974ac32db61526ae` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:13.464593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b032ba8bfcc54f1d974ac32db61526ae)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:13.466704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:13.698772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b032ba8bfcc54f1d974ac32db61526ae'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:13.759181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b032ba8bfcc54f1d974ac32db61526ae'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:13.759822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b032ba8bfcc54f1d974ac32db61526ae'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:13.760451 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b032ba8bfcc54f1d974ac32db61526ae'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:13.768309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b032ba8bfcc54f1d974ac32db61526ae'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:13.788530 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1019/2064] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:13 2022] DELETE /identity/v3/users/b032ba8bfcc54f1d974ac32db61526ae => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:13.807356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:14.062155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:14.063786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:14.077870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4a634-f6bb-4f65-8afa-da79fa4e0029 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:14.136202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b3b01-469d-4240-a962-926b4344d558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:14.136842 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b3b01-469d-4240-a962-926b4344d558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:14.137123 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b3b01-469d-4240-a962-926b4344d558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:14.137397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b3b01-469d-4240-a962-926b4344d558 admin admin] PATH_INFO: `/v3/users/bf60a530c54946a7b4d62392a1a4ba4a` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:14.138579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b3b01-469d-4240-a962-926b4344d558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf60a530c54946a7b4d62392a1a4ba4a)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:14.140583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b3b01-469d-4240-a962-926b4344d558 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:14.289989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:14.291556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:14.305712 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8acb197-ed14-4820-835a-e860b8753699 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:14.310537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:14.311110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:14.311392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:14.311667 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] PATH_INFO: `/v3/projects/f248d78c873942cab2bb2c57369fe4db` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:14.312819 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f248d78c873942cab2bb2c57369fe4db)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:14.314792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:14.381604 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-040b3b01-469d-4240-a962-926b4344d558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf60a530c54946a7b4d62392a1a4ba4a'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:14.412927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f248d78c873942cab2bb2c57369fe4db', 'request_id': 'req-fc4d0b39-6875-48bc-8e10-852579c556e4'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:14.481508 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-040b3b01-469d-4240-a962-926b4344d558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf60a530c54946a7b4d62392a1a4ba4a'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:14.482176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-040b3b01-469d-4240-a962-926b4344d558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf60a530c54946a7b4d62392a1a4ba4a'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:14.482926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-040b3b01-469d-4240-a962-926b4344d558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf60a530c54946a7b4d62392a1a4ba4a'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:14.491122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-040b3b01-469d-4240-a962-926b4344d558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf60a530c54946a7b4d62392a1a4ba4a'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:14.507954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1046/2065] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:14 2022] DELETE /identity/v3/users/bf60a530c54946a7b4d62392a1a4ba4a => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:14.512444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadfd3c-2109-4c7f-bd96-05e1bf3ad9c7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:14.512732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadfd3c-2109-4c7f-bd96-05e1bf3ad9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:14.513010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadfd3c-2109-4c7f-bd96-05e1bf3ad9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:14.532150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-7dadfd3c-2109-4c7f-bd96-05e1bf3ad9c7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:14.533365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7dadfd3c-2109-4c7f-bd96-05e1bf3ad9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:14.556492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f248d78c873942cab2bb2c57369fe4db', 'request_id': 'req-fc4d0b39-6875-48bc-8e10-852579c556e4'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:14.564502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f248d78c873942cab2bb2c57369fe4db', 'request_id': 'req-fc4d0b39-6875-48bc-8e10-852579c556e4'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:14.578243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] The token cache is being invalidate because project f248d78c873942cab2bb2c57369fe4db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:14.578726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:14.579590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-fc4d0b39-6875-48bc-8e10-852579c556e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:14.582449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1020/2066] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:14 2022] DELETE /identity/v3/projects/f248d78c873942cab2bb2c57369fe4db => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:14.594740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cb55e-7b20-4799-b685-85fcf7bf9ccc None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:14.595030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cb55e-7b20-4799-b685-85fcf7bf9ccc None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:14.595290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cb55e-7b20-4799-b685-85fcf7bf9ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:14.618865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1047/2067] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:14.624800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7b8cb55e-7b20-4799-b685-85fcf7bf9ccc None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:14.626200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7b8cb55e-7b20-4799-b685-85fcf7bf9ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:14.633191 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadfd3c-2109-4c7f-bd96-05e1bf3ad9c7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:14.634879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1021/2068] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:14.637356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7dadfd3c-2109-4c7f-bd96-05e1bf3ad9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:14.644615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdee27e-41a5-45ef-9dc2-c0010ec30165 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:14.645003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdee27e-41a5-45ef-9dc2-c0010ec30165 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:14.645381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdee27e-41a5-45ef-9dc2-c0010ec30165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:14.667062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7bdee27e-41a5-45ef-9dc2-c0010ec30165 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:14.668629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7bdee27e-41a5-45ef-9dc2-c0010ec30165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:14.678041 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1022/2069] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:14.682704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdee27e-41a5-45ef-9dc2-c0010ec30165 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:14.693148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdee27e-41a5-45ef-9dc2-c0010ec30165 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:14.699075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e8370-fe18-4505-8c45-8b6173ccb139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:14.699753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e8370-fe18-4505-8c45-8b6173ccb139 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:14.700105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e8370-fe18-4505-8c45-8b6173ccb139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:14.700386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e8370-fe18-4505-8c45-8b6173ccb139 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:14.701454 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e8370-fe18-4505-8c45-8b6173ccb139 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:14.701576 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e8370-fe18-4505-8c45-8b6173ccb139 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:14.703906 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e8370-fe18-4505-8c45-8b6173ccb139 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:14.733731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1023/2070] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:24:14.738434 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e8370-fe18-4505-8c45-8b6173ccb139 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:14.742235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadfd3c-2109-4c7f-bd96-05e1bf3ad9c7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:14.746563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa7140-5585-4a3f-90d4-5eabf41e33b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:14.747203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa7140-5585-4a3f-90d4-5eabf41e33b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:14.747885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa7140-5585-4a3f-90d4-5eabf41e33b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:14.747885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa7140-5585-4a3f-90d4-5eabf41e33b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:14.750081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e8370-fe18-4505-8c45-8b6173ccb139 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:14.757674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af501c60-b856-48a8-b62a-4a66b14f8c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:14.757868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa7140-5585-4a3f-90d4-5eabf41e33b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:14.758411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501c60-b856-48a8-b62a-4a66b14f8c09 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:14.758757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501c60-b856-48a8-b62a-4a66b14f8c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:14.759148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af501c60-b856-48a8-b62a-4a66b14f8c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:14.759312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa7140-5585-4a3f-90d4-5eabf41e33b9 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:14.759968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af501c60-b856-48a8-b62a-4a66b14f8c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:14.762112 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af501c60-b856-48a8-b62a-4a66b14f8c09 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:14.771203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1048/2071] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:14.926634 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1024/2072] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:14 2022] POST /identity/v3/projects => generated 379 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:14.931033 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa7140-5585-4a3f-90d4-5eabf41e33b9 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:14.941787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa7140-5585-4a3f-90d4-5eabf41e33b9 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:15.012672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cf925-30f9-4764-8d09-bf2f96bc5515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:15.013253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cf925-30f9-4764-8d09-bf2f96bc5515 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:15.013575 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cf925-30f9-4764-8d09-bf2f96bc5515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:15.013856 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86cf925-30f9-4764-8d09-bf2f96bc5515 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:15.016792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cf925-30f9-4764-8d09-bf2f96bc5515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:15.031324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86cf925-30f9-4764-8d09-bf2f96bc5515 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:15.038979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:INSERT=3,keystone:SELECT=84,keystone:DELETE=13 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:15.113513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:15.126714 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1049/2073] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:14 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:15.127092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af501c60-b856-48a8-b62a-4a66b14f8c09 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:15.141786 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af501c60-b856-48a8-b62a-4a66b14f8c09 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:15.148420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3e639-06bd-4d37-926d-0f77a0ba7e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:15.149121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3e639-06bd-4d37-926d-0f77a0ba7e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:15.154081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3e639-06bd-4d37-926d-0f77a0ba7e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:15.154081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3e639-06bd-4d37-926d-0f77a0ba7e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:15.154081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3e639-06bd-4d37-926d-0f77a0ba7e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:15.154081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d3e639-06bd-4d37-926d-0f77a0ba7e6e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:15.169028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cf925-30f9-4764-8d09-bf2f96bc5515 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:15.176128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1025/2074] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:24:15.183963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86cf925-30f9-4764-8d09-bf2f96bc5515 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:15.194075 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca2f98-7a74-494e-99db-44dff29ad449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:15.194945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca2f98-7a74-494e-99db-44dff29ad449 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:15.195426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca2f98-7a74-494e-99db-44dff29ad449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:15.196582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca2f98-7a74-494e-99db-44dff29ad449 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:15.197456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca2f98-7a74-494e-99db-44dff29ad449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:15.200886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca2f98-7a74-494e-99db-44dff29ad449 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:15.215886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1050/2075] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:15.217415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3e639-06bd-4d37-926d-0f77a0ba7e6e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:15.229915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3e639-06bd-4d37-926d-0f77a0ba7e6e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:15.236413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560115e4-bcff-4663-bd51-e8dc2570d70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:15.237290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560115e4-bcff-4663-bd51-e8dc2570d70f admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:15.238156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560115e4-bcff-4663-bd51-e8dc2570d70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:15.238519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560115e4-bcff-4663-bd51-e8dc2570d70f admin admin] PATH_INFO: `/v3/projects/f14c1634c5c44bcb966fc2817a94a6f4/users/c0019e40c05c4b8a8387584fff6e12ea/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:15.257165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560115e4-bcff-4663-bd51-e8dc2570d70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14c1634c5c44bcb966fc2817a94a6f4, user_id=c0019e40c05c4b8a8387584fff6e12ea, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:15.260152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560115e4-bcff-4663-bd51-e8dc2570d70f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:15.398645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1026/2076] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:15 2022] PUT /identity/v3/projects/f14c1634c5c44bcb966fc2817a94a6f4/users/c0019e40c05c4b8a8387584fff6e12ea/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:15.404837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca2f98-7a74-494e-99db-44dff29ad449 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:15.406875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-70ca2f98-7a74-494e-99db-44dff29ad449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:15.407176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30839b-1523-4a94-9601-6da9e680d293 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:15.407501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30839b-1523-4a94-9601-6da9e680d293 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:15.407822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d30839b-1523-4a94-9601-6da9e680d293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:15.433358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca2f98-7a74-494e-99db-44dff29ad449 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:15.445768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-5d30839b-1523-4a94-9601-6da9e680d293 None None] MFA Rules not processed for user `c0019e40c05c4b8a8387584fff6e12ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:15.447123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-5d30839b-1523-4a94-9601-6da9e680d293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:15.529224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:15.530540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:15.530891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:15.531262 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] PATH_INFO: `/v3/projects/005c5a91a8e54f778ed19ff1231f7639` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:15.535109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=005c5a91a8e54f778ed19ff1231f7639)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:15.537491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:15.583227 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1027/2077] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:15.587699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30839b-1523-4a94-9601-6da9e680d293 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:15.599604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d30839b-1523-4a94-9601-6da9e680d293 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:15.604908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0067df-a4ae-4517-8993-8fa0faacf8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:15.605749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0067df-a4ae-4517-8993-8fa0faacf8ee admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:15.605994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0067df-a4ae-4517-8993-8fa0faacf8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:15.606265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0067df-a4ae-4517-8993-8fa0faacf8ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:15.606896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0067df-a4ae-4517-8993-8fa0faacf8ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:15.608828 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0067df-a4ae-4517-8993-8fa0faacf8ee admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:15.653337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '005c5a91a8e54f778ed19ff1231f7639', 'request_id': 'req-55f53ca8-0743-4a01-a45c-c032f3f94cf5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:15.668423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '005c5a91a8e54f778ed19ff1231f7639', 'request_id': 'req-55f53ca8-0743-4a01-a45c-c032f3f94cf5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:15.748996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1028/2078] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:15 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:15.753550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0067df-a4ae-4517-8993-8fa0faacf8ee admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:15.796962 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '005c5a91a8e54f778ed19ff1231f7639', 'request_id': 'req-55f53ca8-0743-4a01-a45c-c032f3f94cf5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:15.814763 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] The token cache is being invalidate because project 005c5a91a8e54f778ed19ff1231f7639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:15.815156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:15.816070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:15.817881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1051/2079] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:15 2022] DELETE /identity/v3/projects/005c5a91a8e54f778ed19ff1231f7639 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:15.821925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:15.824017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:15.867022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0067df-a4ae-4517-8993-8fa0faacf8ee admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:15.886508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea9662-9185-45be-87c3-970fd2735c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:15.887331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea9662-9185-45be-87c3-970fd2735c1a admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:15.887659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea9662-9185-45be-87c3-970fd2735c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:15.887965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ea9662-9185-45be-87c3-970fd2735c1a admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:15.888820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea9662-9185-45be-87c3-970fd2735c1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:15.890924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ea9662-9185-45be-87c3-970fd2735c1a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:15.931578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f53ca8-0743-4a01-a45c-c032f3f94cf5 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:15.937276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:15.937952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:15.938377 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:15.938794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] PATH_INFO: `/v3/users/8959b2186e3a408c858059ccbfc02355` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:15.944668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8959b2186e3a408c858059ccbfc02355)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:15.946622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:16.058107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1029/2080] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:15 2022] POST /identity/v3/users => generated 296 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:16.068018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea9662-9185-45be-87c3-970fd2735c1a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:16.070314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d5ea9662-9185-45be-87c3-970fd2735c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:16.095081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ea9662-9185-45be-87c3-970fd2735c1a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:16.106955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078edc3-db03-48f4-baf0-f5e14f5fd0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:16.107635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078edc3-db03-48f4-baf0-f5e14f5fd0c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:16.108023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078edc3-db03-48f4-baf0-f5e14f5fd0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:16.108240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d078edc3-db03-48f4-baf0-f5e14f5fd0c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:16.108946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078edc3-db03-48f4-baf0-f5e14f5fd0c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:16.110572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d078edc3-db03-48f4-baf0-f5e14f5fd0c1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:16.124998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1030/2081] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:16.126025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078edc3-db03-48f4-baf0-f5e14f5fd0c1 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:16.136544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d078edc3-db03-48f4-baf0-f5e14f5fd0c1 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:16.141977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc696e1-78eb-410f-bd8a-09ecac9e58b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:16.142117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc696e1-78eb-410f-bd8a-09ecac9e58b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:16.142371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc696e1-78eb-410f-bd8a-09ecac9e58b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:16.142662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc696e1-78eb-410f-bd8a-09ecac9e58b9 admin admin] PATH_INFO: `/v3/projects/3271c231153949b4a23a2d0a6f92e3c1/users/6b68fec240c348da85f1a032846e876d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:16.166261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc696e1-78eb-410f-bd8a-09ecac9e58b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3271c231153949b4a23a2d0a6f92e3c1, user_id=6b68fec240c348da85f1a032846e876d, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:16.168863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8959b2186e3a408c858059ccbfc02355'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:16.168983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc696e1-78eb-410f-bd8a-09ecac9e58b9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:16.280257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8959b2186e3a408c858059ccbfc02355'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:16.283023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8959b2186e3a408c858059ccbfc02355'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:16.285167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8959b2186e3a408c858059ccbfc02355'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:16.295749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1031/2082] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:16 2022] PUT /identity/v3/projects/3271c231153949b4a23a2d0a6f92e3c1/users/6b68fec240c348da85f1a032846e876d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:16.295749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc696e1-78eb-410f-bd8a-09ecac9e58b9 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:16.296007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8959b2186e3a408c858059ccbfc02355'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:16.296097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7fc696e1-78eb-410f-bd8a-09ecac9e58b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:16.319285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1052/2083] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:15 2022] DELETE /identity/v3/users/8959b2186e3a408c858059ccbfc02355 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:16.379150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc696e1-78eb-410f-bd8a-09ecac9e58b9 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:16.459237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c6e94-383c-4b05-a174-1790209beb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:16.460542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c6e94-383c-4b05-a174-1790209beb29 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:16.460965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c6e94-383c-4b05-a174-1790209beb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:16.461919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c6e94-383c-4b05-a174-1790209beb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:16.462706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c6e94-383c-4b05-a174-1790209beb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:16.464488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c6e94-383c-4b05-a174-1790209beb29 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:16.489052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1032/2084] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:16.489782 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:16.496591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c6e94-383c-4b05-a174-1790209beb29 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:16.502635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f80ac6-4da6-49bb-a38d-97b6de6dd4b1 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:16.506530 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-994c6e94-383c-4b05-a174-1790209beb29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:16.509710 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e207d-da7d-461b-8871-b5715d4fc248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:16.513719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e207d-da7d-461b-8871-b5715d4fc248 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:16.513719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e207d-da7d-461b-8871-b5715d4fc248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:16.513719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e207d-da7d-461b-8871-b5715d4fc248 admin admin] PATH_INFO: `/v3/projects/3271c231153949b4a23a2d0a6f92e3c1/users/6b68fec240c348da85f1a032846e876d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:16.513719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e207d-da7d-461b-8871-b5715d4fc248 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3271c231153949b4a23a2d0a6f92e3c1, user_id=6b68fec240c348da85f1a032846e876d, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:16.516546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e207d-da7d-461b-8871-b5715d4fc248 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:16.529114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c6e94-383c-4b05-a174-1790209beb29 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:16.538463 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1053/2085] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:16 2022] PUT /identity/v3/projects/3271c231153949b4a23a2d0a6f92e3c1/users/6b68fec240c348da85f1a032846e876d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:16.545743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c8587-5061-4974-926c-aef936b0a425 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:16.546150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c8587-5061-4974-926c-aef936b0a425 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:16.546548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c8587-5061-4974-926c-aef936b0a425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:16.616265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-169c8587-5061-4974-926c-aef936b0a425 None None] MFA Rules not processed for user `6b68fec240c348da85f1a032846e876d`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:16.617530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-169c8587-5061-4974-926c-aef936b0a425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:16.658100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a1d02-2114-43bb-8348-493fb0027cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:16.658997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a1d02-2114-43bb-8348-493fb0027cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:16.658997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a1d02-2114-43bb-8348-493fb0027cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:16.659370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a1d02-2114-43bb-8348-493fb0027cc5 admin admin] PATH_INFO: `/v3/users/a779af5cc55943dd9a8dfd66c0bf6023` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:16.663035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a1d02-2114-43bb-8348-493fb0027cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a779af5cc55943dd9a8dfd66c0bf6023)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:16.664993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a1d02-2114-43bb-8348-493fb0027cc5 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:16.771546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1054/2086] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:16.786132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c8587-5061-4974-926c-aef936b0a425 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:16.790480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-169c8587-5061-4974-926c-aef936b0a425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:16.894301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c8587-5061-4974-926c-aef936b0a425 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:16.896166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8b6a1d02-2114-43bb-8348-493fb0027cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a779af5cc55943dd9a8dfd66c0bf6023'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:16.898555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96067caa-8bbf-4679-88cd-a804722f7a8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:16.899188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96067caa-8bbf-4679-88cd-a804722f7a8d service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:16.899870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96067caa-8bbf-4679-88cd-a804722f7a8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:16.899870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96067caa-8bbf-4679-88cd-a804722f7a8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:16.914312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96067caa-8bbf-4679-88cd-a804722f7a8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:16.914312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96067caa-8bbf-4679-88cd-a804722f7a8d service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:16.923126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1055/2087] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:17.008745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96067caa-8bbf-4679-88cd-a804722f7a8d service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:17.010606 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-96067caa-8bbf-4679-88cd-a804722f7a8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:17.026814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96067caa-8bbf-4679-88cd-a804722f7a8d service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:17.029001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8b6a1d02-2114-43bb-8348-493fb0027cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a779af5cc55943dd9a8dfd66c0bf6023'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:17.031009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8b6a1d02-2114-43bb-8348-493fb0027cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a779af5cc55943dd9a8dfd66c0bf6023'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:17.032430 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8b6a1d02-2114-43bb-8348-493fb0027cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a779af5cc55943dd9a8dfd66c0bf6023'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:17.035089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:17.035089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:17.035089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:17.035089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] PATH_INFO: `/v3/projects/d939f1adaa5545cba7fb7ebbd950a26e` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:17.035970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d939f1adaa5545cba7fb7ebbd950a26e)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:17.037996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:17.047957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8b6a1d02-2114-43bb-8348-493fb0027cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a779af5cc55943dd9a8dfd66c0bf6023'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:17.119174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1033/2088] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:16 2022] DELETE /identity/v3/users/a779af5cc55943dd9a8dfd66c0bf6023 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:17.123681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f084-b479-4564-b8b1-2cf551e27bd4 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:17.124351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f084-b479-4564-b8b1-2cf551e27bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:17.124785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f084-b479-4564-b8b1-2cf551e27bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:17.158459 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-25a2f084-b479-4564-b8b1-2cf551e27bd4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:17.161591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-25a2f084-b479-4564-b8b1-2cf551e27bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:17.173343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd939f1adaa5545cba7fb7ebbd950a26e', 'request_id': 'req-b9647c0f-ffa6-4984-8fb7-7dead402157d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:17.188769 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd939f1adaa5545cba7fb7ebbd950a26e', 'request_id': 'req-b9647c0f-ffa6-4984-8fb7-7dead402157d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:17.255622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd939f1adaa5545cba7fb7ebbd950a26e', 'request_id': 'req-b9647c0f-ffa6-4984-8fb7-7dead402157d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:17.269328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] The token cache is being invalidate because project d939f1adaa5545cba7fb7ebbd950a26e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:17.278273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:17.280326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:17.283681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1056/2089] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:17 2022] DELETE /identity/v3/projects/d939f1adaa5545cba7fb7ebbd950a26e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:17.285907 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:17.288040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:17.294987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1034/2090] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:17.294987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa726a-bb33-423e-8034-01bd349bdc51 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:17.294987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa726a-bb33-423e-8034-01bd349bdc51 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:17.294987 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa726a-bb33-423e-8034-01bd349bdc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:17.325455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-acaa726a-bb33-423e-8034-01bd349bdc51 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:17.326740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-acaa726a-bb33-423e-8034-01bd349bdc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:17.339906 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1035/2091] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:17.341134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa726a-bb33-423e-8034-01bd349bdc51 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:17.347155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-acaa726a-bb33-423e-8034-01bd349bdc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:17.391197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9647c0f-ffa6-4984-8fb7-7dead402157d admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:17.396067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:17.399949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dbb77-7294-4d53-b81f-ebf89bad7700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:17.400707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dbb77-7294-4d53-b81f-ebf89bad7700 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:17.401092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dbb77-7294-4d53-b81f-ebf89bad7700 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:17.402030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dbb77-7294-4d53-b81f-ebf89bad7700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:17.404723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-af3dbb77-7294-4d53-b81f-ebf89bad7700 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:17.435485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa726a-bb33-423e-8034-01bd349bdc51 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:17.455638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a560b3c-b884-44d5-9a6a-6e9d0fcb0705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:17.456483 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560b3c-b884-44d5-9a6a-6e9d0fcb0705 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:17.456952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560b3c-b884-44d5-9a6a-6e9d0fcb0705 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:17.457457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a560b3c-b884-44d5-9a6a-6e9d0fcb0705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:17.468421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:17.473867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a560b3c-b884-44d5-9a6a-6e9d0fcb0705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:17.476319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a560b3c-b884-44d5-9a6a-6e9d0fcb0705 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:17.490053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3dbb77-7294-4d53-b81f-ebf89bad7700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:17.492616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3dbb77-7294-4d53-b81f-ebf89bad7700 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:17.504219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1057/2092] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:17.505690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcebd5-5075-41bc-b9dd-3082aba455b2 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:17.507107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcebd5-5075-41bc-b9dd-3082aba455b2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:17.507107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcebd5-5075-41bc-b9dd-3082aba455b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:17.511150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1036/2093] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:17.529357 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-33fcebd5-5075-41bc-b9dd-3082aba455b2 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:17.531361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-33fcebd5-5075-41bc-b9dd-3082aba455b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:17.541861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1058/2094] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:17.547746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a560b3c-b884-44d5-9a6a-6e9d0fcb0705 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:17.562178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a560b3c-b884-44d5-9a6a-6e9d0fcb0705 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:17.573375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9487d-bcd8-4431-bcfc-465d063c2055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:17.574099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9487d-bcd8-4431-bcfc-465d063c2055 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:17.574481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9487d-bcd8-4431-bcfc-465d063c2055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:17.574853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9487d-bcd8-4431-bcfc-465d063c2055 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:17.575679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9487d-bcd8-4431-bcfc-465d063c2055 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:17.576135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9487d-bcd8-4431-bcfc-465d063c2055 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:17.578006 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9487d-bcd8-4431-bcfc-465d063c2055 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:17.613365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1037/2095] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:24:17.615933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcebd5-5075-41bc-b9dd-3082aba455b2 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:17.630070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcebd5-5075-41bc-b9dd-3082aba455b2 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:17.666001 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ab07c-8e67-4faa-99fe-ba2b65f7a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:17.666646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ab07c-8e67-4faa-99fe-ba2b65f7a5fd admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:17.666910 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ab07c-8e67-4faa-99fe-ba2b65f7a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:17.667180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ab07c-8e67-4faa-99fe-ba2b65f7a5fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:17.667847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ab07c-8e67-4faa-99fe-ba2b65f7a5fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:17.674000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ab07c-8e67-4faa-99fe-ba2b65f7a5fd admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:17.804174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1059/2096] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:17 2022] POST /identity/v3/projects => generated 385 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:17.808576 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9487d-bcd8-4431-bcfc-465d063c2055 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:17.821109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9487d-bcd8-4431-bcfc-465d063c2055 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:17.917172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287997a7-85b3-45ac-9fc6-6190cba9cdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:17.918019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287997a7-85b3-45ac-9fc6-6190cba9cdd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:17.918441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287997a7-85b3-45ac-9fc6-6190cba9cdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:17.918814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287997a7-85b3-45ac-9fc6-6190cba9cdd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:17.919668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287997a7-85b3-45ac-9fc6-6190cba9cdd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:17.922362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287997a7-85b3-45ac-9fc6-6190cba9cdd9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:18.067820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1038/2097] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:17 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:18.073800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ab07c-8e67-4faa-99fe-ba2b65f7a5fd admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:18.086350 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ab07c-8e67-4faa-99fe-ba2b65f7a5fd admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:18.091685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db7b4a-d60d-47f6-843f-9dba86d15e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:18.092348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db7b4a-d60d-47f6-843f-9dba86d15e66 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:18.092722 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db7b4a-d60d-47f6-843f-9dba86d15e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:18.093001 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db7b4a-d60d-47f6-843f-9dba86d15e66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:18.093647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db7b4a-d60d-47f6-843f-9dba86d15e66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:18.095331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db7b4a-d60d-47f6-843f-9dba86d15e66 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:18.107861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1060/2098] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:18.117779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287997a7-85b3-45ac-9fc6-6190cba9cdd9 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:18.128869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287997a7-85b3-45ac-9fc6-6190cba9cdd9 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:18.134635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d71a9-851a-4322-9513-7e5e5d38b3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:18.135276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d71a9-851a-4322-9513-7e5e5d38b3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:18.135558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d71a9-851a-4322-9513-7e5e5d38b3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:18.136024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d71a9-851a-4322-9513-7e5e5d38b3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:18.136680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d71a9-851a-4322-9513-7e5e5d38b3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:18.138326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d71a9-851a-4322-9513-7e5e5d38b3a5 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:18.156304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db7b4a-d60d-47f6-843f-9dba86d15e66 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:18.157542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1039/2099] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:18.166792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db7b4a-d60d-47f6-843f-9dba86d15e66 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:18.172540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0617ec1-1313-4554-adc0-a9e298385041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:18.173208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0617ec1-1313-4554-adc0-a9e298385041 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:18.173484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0617ec1-1313-4554-adc0-a9e298385041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:18.173754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0617ec1-1313-4554-adc0-a9e298385041 admin admin] PATH_INFO: `/v3/projects/46b6a5bac5ac4cf3a203992e2c714b0a/users/3f716e79326244ed8d14f803ae9f3d64/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:18.192296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0617ec1-1313-4554-adc0-a9e298385041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46b6a5bac5ac4cf3a203992e2c714b0a, user_id=3f716e79326244ed8d14f803ae9f3d64, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:18.194947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0617ec1-1313-4554-adc0-a9e298385041 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:18.256729 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d71a9-851a-4322-9513-7e5e5d38b3a5 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:18.274051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-706d71a9-851a-4322-9513-7e5e5d38b3a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:18.282907 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1061/2100] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:18 2022] PUT /identity/v3/projects/46b6a5bac5ac4cf3a203992e2c714b0a/users/3f716e79326244ed8d14f803ae9f3d64/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:18.299461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6048d12-fce4-4b2c-bd74-c7a9e7344b2c None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:18.300294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6048d12-fce4-4b2c-bd74-c7a9e7344b2c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:18.300568 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6048d12-fce4-4b2c-bd74-c7a9e7344b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:18.322792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d71a9-851a-4322-9513-7e5e5d38b3a5 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:18.340152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-a6048d12-fce4-4b2c-bd74-c7a9e7344b2c None None] MFA Rules not processed for user `3f716e79326244ed8d14f803ae9f3d64`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:18.342221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a6048d12-fce4-4b2c-bd74-c7a9e7344b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:18.415265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:18.416092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:18.416903 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:18.417368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] PATH_INFO: `/v3/projects/61d2531e30904c9cafac2cc80a74cf9b` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:18.419166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] RBAC: Authorizing `identity:delete_project(project_id=61d2531e30904c9cafac2cc80a74cf9b)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:18.424384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:18.433146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1062/2101] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:18.447445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6048d12-fce4-4b2c-bd74-c7a9e7344b2c None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:18.449514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a6048d12-fce4-4b2c-bd74-c7a9e7344b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:18.524431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6048d12-fce4-4b2c-bd74-c7a9e7344b2c None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:18.528527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e511d2-2b82-4b1a-b97e-7e26189ceee4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:18.529161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e511d2-2b82-4b1a-b97e-7e26189ceee4 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:18.529442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e511d2-2b82-4b1a-b97e-7e26189ceee4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:18.529723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e511d2-2b82-4b1a-b97e-7e26189ceee4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:18.540020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e511d2-2b82-4b1a-b97e-7e26189ceee4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:18.541452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e511d2-2b82-4b1a-b97e-7e26189ceee4 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:18.552072 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1063/2102] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:24:18.563392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61d2531e30904c9cafac2cc80a74cf9b', 'request_id': 'req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:18.573077 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e511d2-2b82-4b1a-b97e-7e26189ceee4 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:18.583996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e511d2-2b82-4b1a-b97e-7e26189ceee4 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:18.642302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61d2531e30904c9cafac2cc80a74cf9b', 'request_id': 'req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:18.656211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61d2531e30904c9cafac2cc80a74cf9b', 'request_id': 'req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:18.675512 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] The token cache is being invalidate because project 61d2531e30904c9cafac2cc80a74cf9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:18.675944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:18.682411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:18.682740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:18.682740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:18.682740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:18.682740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] PATH_INFO: `/v3/users/3f716e79326244ed8d14f803ae9f3d64` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:18.683461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1040/2103] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:18 2022] DELETE /identity/v3/projects/61d2531e30904c9cafac2cc80a74cf9b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:18.691880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:18.692975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f716e79326244ed8d14f803ae9f3d64)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:18.694421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:18.695128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:18.714521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63f7a08-caa9-4f5d-ab70-ec5cece8806d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:18.720788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08a335-6def-4f67-8065-3ffa7ae29bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:18.722092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08a335-6def-4f67-8065-3ffa7ae29bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:18.722428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08a335-6def-4f67-8065-3ffa7ae29bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:18.722887 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08a335-6def-4f67-8065-3ffa7ae29bf7 admin admin] PATH_INFO: `/v3/users/9fddccd9808e4e30af0342e5ffb56ac7` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:18.724500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08a335-6def-4f67-8065-3ffa7ae29bf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fddccd9808e4e30af0342e5ffb56ac7)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:18.726799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08a335-6def-4f67-8065-3ffa7ae29bf7 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:18.919140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-0a08a335-6def-4f67-8065-3ffa7ae29bf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fddccd9808e4e30af0342e5ffb56ac7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:18.923240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f716e79326244ed8d14f803ae9f3d64'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:18.984234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-0a08a335-6def-4f67-8065-3ffa7ae29bf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fddccd9808e4e30af0342e5ffb56ac7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:18.984978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-0a08a335-6def-4f67-8065-3ffa7ae29bf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fddccd9808e4e30af0342e5ffb56ac7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:18.985783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-0a08a335-6def-4f67-8065-3ffa7ae29bf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fddccd9808e4e30af0342e5ffb56ac7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:19.000788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-0a08a335-6def-4f67-8065-3ffa7ae29bf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fddccd9808e4e30af0342e5ffb56ac7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:19.039803 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1041/2104] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:18 2022] DELETE /identity/v3/users/9fddccd9808e4e30af0342e5ffb56ac7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:19.082335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f716e79326244ed8d14f803ae9f3d64'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:19.084834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f716e79326244ed8d14f803ae9f3d64'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:19.087060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f716e79326244ed8d14f803ae9f3d64'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:19.113561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f716e79326244ed8d14f803ae9f3d64'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:19.165884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1064/2105] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:18 2022] DELETE /identity/v3/users/3f716e79326244ed8d14f803ae9f3d64 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:19.174293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fbc73-0020-45c4-932f-48901b418112 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:19.174608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fbc73-0020-45c4-932f-48901b418112 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:19.174889 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fbc73-0020-45c4-932f-48901b418112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:19.205336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-fb9fbc73-0020-45c4-932f-48901b418112 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:19.207118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-fb9fbc73-0020-45c4-932f-48901b418112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:19.300820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1042/2106] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:19.315960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:19.323629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:19.325534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:19.438933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ed224-1666-4555-8747-9f1328edcce1 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:19.445645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc779466-0c1d-4600-b90a-f683537355d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:19.446336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc779466-0c1d-4600-b90a-f683537355d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:19.446665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc779466-0c1d-4600-b90a-f683537355d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:19.446991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc779466-0c1d-4600-b90a-f683537355d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:19.461289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc779466-0c1d-4600-b90a-f683537355d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:19.464139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc779466-0c1d-4600-b90a-f683537355d0 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:19.491896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1065/2107] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:19.570171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fbc73-0020-45c4-932f-48901b418112 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:19.571790 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-fb9fbc73-0020-45c4-932f-48901b418112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:19.609697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fbc73-0020-45c4-932f-48901b418112 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:19.641760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:19.642480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:19.642769 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:19.643053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] PATH_INFO: `/v3/projects/3b4019c2603543a0a01f956d39a67055` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:19.645414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b4019c2603543a0a01f956d39a67055)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:19.647354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:19.879486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:19.881985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b4019c2603543a0a01f956d39a67055', 'request_id': 'req-99f68a53-c0fd-4d21-a064-84743cc1cb0e'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:19.989832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b4019c2603543a0a01f956d39a67055', 'request_id': 'req-99f68a53-c0fd-4d21-a064-84743cc1cb0e'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:20.008288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b4019c2603543a0a01f956d39a67055', 'request_id': 'req-99f68a53-c0fd-4d21-a064-84743cc1cb0e'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:20.022691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] The token cache is being invalidate because project 3b4019c2603543a0a01f956d39a67055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:20.023252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:20.024053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-99f68a53-c0fd-4d21-a064-84743cc1cb0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:20.025870 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1043/2108] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:19 2022] DELETE /identity/v3/projects/3b4019c2603543a0a01f956d39a67055 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:20.031286 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454fb53-2e68-4766-ad18-c41573310e27 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.031821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454fb53-2e68-4766-ad18-c41573310e27 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.032247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454fb53-2e68-4766-ad18-c41573310e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.060729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-c454fb53-2e68-4766-ad18-c41573310e27 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:20.062600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c454fb53-2e68-4766-ad18-c41573310e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:20.101293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f83cae-4169-46f4-a06b-3b0a96d07c77 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.101654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f83cae-4169-46f4-a06b-3b0a96d07c77 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.101999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f83cae-4169-46f4-a06b-3b0a96d07c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.146617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-d7f83cae-4169-46f4-a06b-3b0a96d07c77 None None] MFA Rules not processed for user `d3da95b627b74c28b921586d0da89521`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:20.148325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d7f83cae-4169-46f4-a06b-3b0a96d07c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:20.189688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1066/2109] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:20.198645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c03031-63bd-4d6b-b13b-f697ebfbf419 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.199201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c03031-63bd-4d6b-b13b-f697ebfbf419 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.199801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c03031-63bd-4d6b-b13b-f697ebfbf419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.227836 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b6c03031-63bd-4d6b-b13b-f697ebfbf419 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:20.230153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b6c03031-63bd-4d6b-b13b-f697ebfbf419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:20.250674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1067/2110] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:20.253779 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c03031-63bd-4d6b-b13b-f697ebfbf419 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:20.267908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1044/2111] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:20.268489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c03031-63bd-4d6b-b13b-f697ebfbf419 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:20.269747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ef3ff-3f52-43a7-b84a-78e3fb6834ce None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.270089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ef3ff-3f52-43a7-b84a-78e3fb6834ce None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.271058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ef3ff-3f52-43a7-b84a-78e3fb6834ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.274936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabff813-ecf3-429e-9d07-64e64f9715eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:20.275665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabff813-ecf3-429e-9d07-64e64f9715eb admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.276229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabff813-ecf3-429e-9d07-64e64f9715eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.285699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabff813-ecf3-429e-9d07-64e64f9715eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.285699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabff813-ecf3-429e-9d07-64e64f9715eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:20.285699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabff813-ecf3-429e-9d07-64e64f9715eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:20.285699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabff813-ecf3-429e-9d07-64e64f9715eb admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:20.294851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1068/2112] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:24:20.296453 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabff813-ecf3-429e-9d07-64e64f9715eb admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:20.301684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fabff813-ecf3-429e-9d07-64e64f9715eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:20.304067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-a44ef3ff-3f52-43a7-b84a-78e3fb6834ce None None] MFA Rules not processed for user `d3da95b627b74c28b921586d0da89521`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:20.305725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a44ef3ff-3f52-43a7-b84a-78e3fb6834ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:20.315152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1045/2113] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:20.316043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ef3ff-3f52-43a7-b84a-78e3fb6834ce None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:20.334158 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ef3ff-3f52-43a7-b84a-78e3fb6834ce None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:20.334340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabff813-ecf3-429e-9d07-64e64f9715eb admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:20.338922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6ad9d-68cf-4bd9-88fb-0a8697cf3328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:20.339537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6ad9d-68cf-4bd9-88fb-0a8697cf3328 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.339853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6ad9d-68cf-4bd9-88fb-0a8697cf3328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.340134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6ad9d-68cf-4bd9-88fb-0a8697cf3328 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.340776 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6ad9d-68cf-4bd9-88fb-0a8697cf3328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:20.342363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:20.342363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.342363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.342616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] PATH_INFO: `/v3/projects/46b6a5bac5ac4cf3a203992e2c714b0a` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.342708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6ad9d-68cf-4bd9-88fb-0a8697cf3328 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:20.343849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=46b6a5bac5ac4cf3a203992e2c714b0a)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:20.345758 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:20.455545 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1046/2114] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:20 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:20.456976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6ad9d-68cf-4bd9-88fb-0a8697cf3328 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:20.460742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-bac6ad9d-68cf-4bd9-88fb-0a8697cf3328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:20.471178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46b6a5bac5ac4cf3a203992e2c714b0a', 'request_id': 'req-47c731b9-0b30-485e-b7d1-4678efa0f1ce'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:20.478474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46b6a5bac5ac4cf3a203992e2c714b0a', 'request_id': 'req-47c731b9-0b30-485e-b7d1-4678efa0f1ce'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:20.542129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6ad9d-68cf-4bd9-88fb-0a8697cf3328 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:20.549742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfc095-5399-450c-acad-413474fdba5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:20.550372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfc095-5399-450c-acad-413474fdba5b service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.550652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfc095-5399-450c-acad-413474fdba5b service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.550927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bfc095-5399-450c-acad-413474fdba5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.606862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46b6a5bac5ac4cf3a203992e2c714b0a', 'request_id': 'req-47c731b9-0b30-485e-b7d1-4678efa0f1ce'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:20.612057 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfc095-5399-450c-acad-413474fdba5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:20.612057 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bfc095-5399-450c-acad-413474fdba5b service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:20.621650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] The token cache is being invalidate because project 46b6a5bac5ac4cf3a203992e2c714b0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:20.622110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1047/2115] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:20 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:20.622718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfc095-5399-450c-acad-413474fdba5b service nova] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:20.633365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:20.635089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-47c731b9-0b30-485e-b7d1-4678efa0f1ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:20.640951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1069/2116] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:20 2022] DELETE /identity/v3/projects/46b6a5bac5ac4cf3a203992e2c714b0a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:20.642157 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22ea3-569e-420c-bff9-efee8aa53bb0 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.642440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22ea3-569e-420c-bff9-efee8aa53bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.642708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22ea3-569e-420c-bff9-efee8aa53bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.663925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-bdc22ea3-569e-420c-bff9-efee8aa53bb0 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:20.665141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-bdc22ea3-569e-420c-bff9-efee8aa53bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:20.699563 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bfc095-5399-450c-acad-413474fdba5b service nova] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:20.705318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623e08c-c265-4607-b278-0ef2b13d9d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:20.709466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623e08c-c265-4607-b278-0ef2b13d9d1f admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.709466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623e08c-c265-4607-b278-0ef2b13d9d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.709466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623e08c-c265-4607-b278-0ef2b13d9d1f admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.709466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623e08c-c265-4607-b278-0ef2b13d9d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:20.710000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623e08c-c265-4607-b278-0ef2b13d9d1f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:20.733955 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1070/2117] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:20.741423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c91c56-fac2-484e-be9a-5d701b1fff6c None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.741713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c91c56-fac2-484e-be9a-5d701b1fff6c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.741984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c91c56-fac2-484e-be9a-5d701b1fff6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.762473 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e2c91c56-fac2-484e-be9a-5d701b1fff6c None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:20.763709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e2c91c56-fac2-484e-be9a-5d701b1fff6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:20.772497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1071/2118] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:20.779886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c91c56-fac2-484e-be9a-5d701b1fff6c None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:20.788972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1048/2119] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:20 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:20.789147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c91c56-fac2-484e-be9a-5d701b1fff6c None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:20.790101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623e08c-c265-4607-b278-0ef2b13d9d1f admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:20.791801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c623e08c-c265-4607-b278-0ef2b13d9d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:20.792401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb1e6f-4703-421b-b1e8-50cdcc4a50c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:20.792952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb1e6f-4703-421b-b1e8-50cdcc4a50c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.793230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb1e6f-4703-421b-b1e8-50cdcc4a50c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.793535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb1e6f-4703-421b-b1e8-50cdcc4a50c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.794205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb1e6f-4703-421b-b1e8-50cdcc4a50c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:20.794507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb1e6f-4703-421b-b1e8-50cdcc4a50c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:20.796390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb1e6f-4703-421b-b1e8-50cdcc4a50c2 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:20.805300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623e08c-c265-4607-b278-0ef2b13d9d1f admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:20.813439 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1072/2120] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:24:20.813611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b7e75-7844-4287-beee-2ce6dfa5c42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:20.813611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b7e75-7844-4287-beee-2ce6dfa5c42e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.813611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b7e75-7844-4287-beee-2ce6dfa5c42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.813611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b7e75-7844-4287-beee-2ce6dfa5c42e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.814029 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b7e75-7844-4287-beee-2ce6dfa5c42e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:20.814029 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b7e75-7844-4287-beee-2ce6dfa5c42e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:20.814810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb1e6f-4703-421b-b1e8-50cdcc4a50c2 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:20.826175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1049/2121] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:20.828808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb1e6f-4703-421b-b1e8-50cdcc4a50c2 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:20.829839 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b7e75-7844-4287-beee-2ce6dfa5c42e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:20.838372 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1223a-4e97-4562-8cef-541f002f5cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:20.839067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1223a-4e97-4562-8cef-541f002f5cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.839404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1223a-4e97-4562-8cef-541f002f5cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.839931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1223a-4e97-4562-8cef-541f002f5cd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.843884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1223a-4e97-4562-8cef-541f002f5cd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:20.843884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1223a-4e97-4562-8cef-541f002f5cd7 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:20.844451 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b7e75-7844-4287-beee-2ce6dfa5c42e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:20.850504 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affddc00-4428-4c98-a62d-19560acb8f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:20.851518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affddc00-4428-4c98-a62d-19560acb8f41 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.852018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affddc00-4428-4c98-a62d-19560acb8f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.852465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affddc00-4428-4c98-a62d-19560acb8f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.853282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affddc00-4428-4c98-a62d-19560acb8f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:20.855163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affddc00-4428-4c98-a62d-19560acb8f41 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:20.872988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:20.872988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1050/2122] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:20.872988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affddc00-4428-4c98-a62d-19560acb8f41 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:20.884139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affddc00-4428-4c98-a62d-19560acb8f41 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:20.888914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531946b1-7487-42d7-8cdd-b52c9d2310c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:20.889598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531946b1-7487-42d7-8cdd-b52c9d2310c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:20.889881 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531946b1-7487-42d7-8cdd-b52c9d2310c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:20.890158 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531946b1-7487-42d7-8cdd-b52c9d2310c4 admin admin] PATH_INFO: `/v3/projects/4aeda7b6b8524f21ab966ab08f77a2d7/users/ca6da8e3af1b43da992a378388200dff/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:20.908546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531946b1-7487-42d7-8cdd-b52c9d2310c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4aeda7b6b8524f21ab966ab08f77a2d7, user_id=ca6da8e3af1b43da992a378388200dff, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:20.911515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531946b1-7487-42d7-8cdd-b52c9d2310c4 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:20.925790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1073/2123] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:20 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:20.931630 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1223a-4e97-4562-8cef-541f002f5cd7 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:20.997628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1223a-4e97-4562-8cef-541f002f5cd7 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:21.002388 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1051/2124] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:20 2022] PUT /identity/v3/projects/4aeda7b6b8524f21ab966ab08f77a2d7/users/ca6da8e3af1b43da992a378388200dff/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:21.016363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b642cb-b59f-404a-9745-9501798a670d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.016726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b642cb-b59f-404a-9745-9501798a670d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.017106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b642cb-b59f-404a-9745-9501798a670d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.048962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-43b642cb-b59f-404a-9745-9501798a670d None None] MFA Rules not processed for user `ca6da8e3af1b43da992a378388200dff`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:21.050320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-43b642cb-b59f-404a-9745-9501798a670d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:21.081429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c138f5-103e-479f-babb-1922cd086aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:21.082137 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c138f5-103e-479f-babb-1922cd086aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.082679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c138f5-103e-479f-babb-1922cd086aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.083887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c138f5-103e-479f-babb-1922cd086aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.084687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c138f5-103e-479f-babb-1922cd086aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:21.086894 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c138f5-103e-479f-babb-1922cd086aa5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:21.131807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1052/2125] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:21.135855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b642cb-b59f-404a-9745-9501798a670d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:21.148352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b642cb-b59f-404a-9745-9501798a670d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:21.154606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed239c-f9c5-4ea1-ac1e-4ba59b4e6ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:21.155352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed239c-f9c5-4ea1-ac1e-4ba59b4e6ebb admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.155953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed239c-f9c5-4ea1-ac1e-4ba59b4e6ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.156335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed239c-f9c5-4ea1-ac1e-4ba59b4e6ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.157073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed239c-f9c5-4ea1-ac1e-4ba59b4e6ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:21.159198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed239c-f9c5-4ea1-ac1e-4ba59b4e6ebb admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:21.195529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1074/2126] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:20 2022] POST /identity/v3/users => generated 287 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:21.198216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c138f5-103e-479f-babb-1922cd086aa5 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:21.209037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c138f5-103e-479f-babb-1922cd086aa5 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:21.214099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d33465-d070-413a-92b8-fc12132a64c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:21.214966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d33465-d070-413a-92b8-fc12132a64c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.215352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d33465-d070-413a-92b8-fc12132a64c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.215728 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d33465-d070-413a-92b8-fc12132a64c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.216507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d33465-d070-413a-92b8-fc12132a64c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:21.218354 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d33465-d070-413a-92b8-fc12132a64c4 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:21.230400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1075/2127] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:24:21.233987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d33465-d070-413a-92b8-fc12132a64c4 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:21.244685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d33465-d070-413a-92b8-fc12132a64c4 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:21.262233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa836156-921a-4fae-a292-9c1cd2b6d675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:21.264155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa836156-921a-4fae-a292-9c1cd2b6d675 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.265233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa836156-921a-4fae-a292-9c1cd2b6d675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.266109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa836156-921a-4fae-a292-9c1cd2b6d675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.267009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa836156-921a-4fae-a292-9c1cd2b6d675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:21.268869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa836156-921a-4fae-a292-9c1cd2b6d675 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:21.277664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1053/2128] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:21 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:21.281741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed239c-f9c5-4ea1-ac1e-4ba59b4e6ebb admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:21.284095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1076/2129] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:24:21.285061 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa836156-921a-4fae-a292-9c1cd2b6d675 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:21.296009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa836156-921a-4fae-a292-9c1cd2b6d675 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:21.372974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed239c-f9c5-4ea1-ac1e-4ba59b4e6ebb admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:21.375370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14205f-2eb1-4f7a-a811-8888b92e8e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:21.376086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14205f-2eb1-4f7a-a811-8888b92e8e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.376370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14205f-2eb1-4f7a-a811-8888b92e8e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.376652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14205f-2eb1-4f7a-a811-8888b92e8e68 admin admin] PATH_INFO: `/v3/projects/4e0ec516b85c4e5db232a791b9fa5134/users/99e43e7859df4a979e15532628d705a0/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.379020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012e4a9-471d-4be5-a9c9-2c3dd22e9171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:21.380022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012e4a9-471d-4be5-a9c9-2c3dd22e9171 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.380022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012e4a9-471d-4be5-a9c9-2c3dd22e9171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.380389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012e4a9-471d-4be5-a9c9-2c3dd22e9171 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.381287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012e4a9-471d-4be5-a9c9-2c3dd22e9171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:21.383526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012e4a9-471d-4be5-a9c9-2c3dd22e9171 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:21.393956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14205f-2eb1-4f7a-a811-8888b92e8e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0ec516b85c4e5db232a791b9fa5134, user_id=99e43e7859df4a979e15532628d705a0, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:21.396827 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14205f-2eb1-4f7a-a811-8888b92e8e68 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:21.470527 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1054/2130] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:21 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:21.474285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012e4a9-471d-4be5-a9c9-2c3dd22e9171 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:21.484467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012e4a9-471d-4be5-a9c9-2c3dd22e9171 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:21.495267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f13039-d042-4189-bc93-65dfc427f551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:21.498163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f13039-d042-4189-bc93-65dfc427f551 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.498163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f13039-d042-4189-bc93-65dfc427f551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.498163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f13039-d042-4189-bc93-65dfc427f551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.498163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f13039-d042-4189-bc93-65dfc427f551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:21.498753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f13039-d042-4189-bc93-65dfc427f551 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:21.506110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1077/2131] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:21 2022] PUT /identity/v3/projects/4e0ec516b85c4e5db232a791b9fa5134/users/99e43e7859df4a979e15532628d705a0/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:21.514146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1055/2132] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:24:21.516911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de41305-de00-49b8-9d14-33cc289dc10f None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.517180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f13039-d042-4189-bc93-65dfc427f551 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:21.517283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de41305-de00-49b8-9d14-33cc289dc10f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.517446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de41305-de00-49b8-9d14-33cc289dc10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.528607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f13039-d042-4189-bc93-65dfc427f551 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:21.550254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1de41305-de00-49b8-9d14-33cc289dc10f None None] MFA Rules not processed for user `99e43e7859df4a979e15532628d705a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:21.550704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1de41305-de00-49b8-9d14-33cc289dc10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:21.571726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:21.613336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7897b-c7d4-478b-9379-d97881a49e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:21.626190 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7897b-c7d4-478b-9379-d97881a49e08 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.626482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7897b-c7d4-478b-9379-d97881a49e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.632934 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7897b-c7d4-478b-9379-d97881a49e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.632934 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b7897b-c7d4-478b-9379-d97881a49e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:21.635144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b7897b-c7d4-478b-9379-d97881a49e08 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:21.660898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1056/2133] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:24:21.666407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7897b-c7d4-478b-9379-d97881a49e08 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:21.678603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7897b-c7d4-478b-9379-d97881a49e08 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:21.684179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43735f-80d3-4a7c-8ecd-5cb0ba797235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:21.685100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43735f-80d3-4a7c-8ecd-5cb0ba797235 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.685606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43735f-80d3-4a7c-8ecd-5cb0ba797235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.686153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43735f-80d3-4a7c-8ecd-5cb0ba797235 admin admin] PATH_INFO: `/v3/projects/bff08c77760945729ee8517d6de31ce5/users/52a6f19e905344fcabc3e7e5084ca9a5/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.698331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1078/2134] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:21.713245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de41305-de00-49b8-9d14-33cc289dc10f None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:21.715045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1de41305-de00-49b8-9d14-33cc289dc10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:21.718311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43735f-80d3-4a7c-8ecd-5cb0ba797235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff08c77760945729ee8517d6de31ce5, user_id=52a6f19e905344fcabc3e7e5084ca9a5, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:21.723806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43735f-80d3-4a7c-8ecd-5cb0ba797235 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:21.824603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de41305-de00-49b8-9d14-33cc289dc10f None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:21.829669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae9725d-220c-44af-9dc0-6cec37695ee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:21.831624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9725d-220c-44af-9dc0-6cec37695ee2 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.832146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9725d-220c-44af-9dc0-6cec37695ee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.833813 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae9725d-220c-44af-9dc0-6cec37695ee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.836171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1057/2135] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:21 2022] PUT /identity/v3/projects/bff08c77760945729ee8517d6de31ce5/users/52a6f19e905344fcabc3e7e5084ca9a5/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:21.837718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04356bc4-fdd3-417a-993f-a676b467a954 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.838009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04356bc4-fdd3-417a-993f-a676b467a954 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.838285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04356bc4-fdd3-417a-993f-a676b467a954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.843960 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9725d-220c-44af-9dc0-6cec37695ee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:21.846017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae9725d-220c-44af-9dc0-6cec37695ee2 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:21.858956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1079/2136] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:24:21.860893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81ca01-e9ad-44b9-b6d5-e1d473c5d9a2 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:21.861180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81ca01-e9ad-44b9-b6d5-e1d473c5d9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:21.861454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81ca01-e9ad-44b9-b6d5-e1d473c5d9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:21.892991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-04356bc4-fdd3-417a-993f-a676b467a954 None None] MFA Rules not processed for user `d3da95b627b74c28b921586d0da89521`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:21.894261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-04356bc4-fdd3-417a-993f-a676b467a954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:21.938643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6c81ca01-e9ad-44b9-b6d5-e1d473c5d9a2 None None] MFA Rules not processed for user `52a6f19e905344fcabc3e7e5084ca9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:21.941016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6c81ca01-e9ad-44b9-b6d5-e1d473c5d9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:21.955130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1058/2137] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:21.956198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04356bc4-fdd3-417a-993f-a676b467a954 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:22.031163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1080/2138] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:22.053159 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81ca01-e9ad-44b9-b6d5-e1d473c5d9a2 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:22.071331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04356bc4-fdd3-417a-993f-a676b467a954 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:22.077745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81ca01-e9ad-44b9-b6d5-e1d473c5d9a2 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:22.081331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9e61d-40ee-475b-a0f8-91b8c7cc95e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:22.081983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9e61d-40ee-475b-a0f8-91b8c7cc95e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:22.082261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9e61d-40ee-475b-a0f8-91b8c7cc95e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:22.082550 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9e61d-40ee-475b-a0f8-91b8c7cc95e7 admin admin] PATH_INFO: `/v3/users/99e43e7859df4a979e15532628d705a0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:22.084095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9e61d-40ee-475b-a0f8-91b8c7cc95e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e43e7859df4a979e15532628d705a0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:22.086030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9e61d-40ee-475b-a0f8-91b8c7cc95e7 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:22.091936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f0552-58a1-453f-bf7d-778f34e4428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:22.092578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f0552-58a1-453f-bf7d-778f34e4428e admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:22.092859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f0552-58a1-453f-bf7d-778f34e4428e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:22.093122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f0552-58a1-453f-bf7d-778f34e4428e admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:22.093768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f0552-58a1-453f-bf7d-778f34e4428e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:22.095695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f0552-58a1-453f-bf7d-778f34e4428e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:22.213412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:22.228546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1081/2139] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:22 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:22.232705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f0552-58a1-453f-bf7d-778f34e4428e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:22.307210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4df9e61d-40ee-475b-a0f8-91b8c7cc95e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e43e7859df4a979e15532628d705a0'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:22.325139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f0552-58a1-453f-bf7d-778f34e4428e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:22.331114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b2058-dd03-4fd6-9bf0-94a40aa3fe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:22.331688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b2058-dd03-4fd6-9bf0-94a40aa3fe13 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:22.332021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b2058-dd03-4fd6-9bf0-94a40aa3fe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:22.332298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b2058-dd03-4fd6-9bf0-94a40aa3fe13 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:22.332944 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b2058-dd03-4fd6-9bf0-94a40aa3fe13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:22.334896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b2058-dd03-4fd6-9bf0-94a40aa3fe13 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:22.436238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4df9e61d-40ee-475b-a0f8-91b8c7cc95e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e43e7859df4a979e15532628d705a0'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:22.437290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4df9e61d-40ee-475b-a0f8-91b8c7cc95e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e43e7859df4a979e15532628d705a0'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:22.438159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4df9e61d-40ee-475b-a0f8-91b8c7cc95e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e43e7859df4a979e15532628d705a0'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:22.448443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-4df9e61d-40ee-475b-a0f8-91b8c7cc95e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e43e7859df4a979e15532628d705a0'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:22.459322 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1082/2140] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:22 2022] POST /identity/v3/users => generated 299 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:22.460301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b2058-dd03-4fd6-9bf0-94a40aa3fe13 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:22.480865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b66b2058-dd03-4fd6-9bf0-94a40aa3fe13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:22.485470 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1059/2141] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:21 2022] DELETE /identity/v3/users/99e43e7859df4a979e15532628d705a0 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:22.490744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540c335-3f2a-4d8a-bda7-8217295a1d36 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:22.491037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540c335-3f2a-4d8a-bda7-8217295a1d36 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:22.491323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540c335-3f2a-4d8a-bda7-8217295a1d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:22.494720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b2058-dd03-4fd6-9bf0-94a40aa3fe13 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:22.517380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0540c335-3f2a-4d8a-bda7-8217295a1d36 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:22.519356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0540c335-3f2a-4d8a-bda7-8217295a1d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:22.589539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a86fc1-bb96-42a2-8b35-f99012dc45fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:22.590097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a86fc1-bb96-42a2-8b35-f99012dc45fd admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:22.590456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a86fc1-bb96-42a2-8b35-f99012dc45fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:22.591564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a86fc1-bb96-42a2-8b35-f99012dc45fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:22.592458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a86fc1-bb96-42a2-8b35-f99012dc45fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:22.594174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a86fc1-bb96-42a2-8b35-f99012dc45fd admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:22.610876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1083/2142] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:22.613192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a86fc1-bb96-42a2-8b35-f99012dc45fd admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:22.627316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1060/2143] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:22.638705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540c335-3f2a-4d8a-bda7-8217295a1d36 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:22.639373 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a86fc1-bb96-42a2-8b35-f99012dc45fd admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:22.640391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0540c335-3f2a-4d8a-bda7-8217295a1d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:22.663247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b04da7-ef0e-4740-af74-20bed6620814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:22.663963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b04da7-ef0e-4740-af74-20bed6620814 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:22.664246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b04da7-ef0e-4740-af74-20bed6620814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:22.664528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b04da7-ef0e-4740-af74-20bed6620814 admin admin] PATH_INFO: `/v3/projects/42628cea306d46588e875127f6f958b8/users/0b5fbd4828104cf99918ec9dcf643297/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:22.687656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b04da7-ef0e-4740-af74-20bed6620814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42628cea306d46588e875127f6f958b8, user_id=0b5fbd4828104cf99918ec9dcf643297, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:22.691651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b04da7-ef0e-4740-af74-20bed6620814 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:22.726240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540c335-3f2a-4d8a-bda7-8217295a1d36 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:22.730072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ac39a-32ee-4f5d-8ee3-41154e6a4832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:22.730711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ac39a-32ee-4f5d-8ee3-41154e6a4832 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:22.731128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ac39a-32ee-4f5d-8ee3-41154e6a4832 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:22.731332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ac39a-32ee-4f5d-8ee3-41154e6a4832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:22.743211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ac39a-32ee-4f5d-8ee3-41154e6a4832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:22.744712 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ac39a-32ee-4f5d-8ee3-41154e6a4832 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:22.755381 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1061/2144] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:24:22.769401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1084/2145] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:22 2022] PUT /identity/v3/projects/42628cea306d46588e875127f6f958b8/users/0b5fbd4828104cf99918ec9dcf643297/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:22.774012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ac39a-32ee-4f5d-8ee3-41154e6a4832 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:22.788800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ac39a-32ee-4f5d-8ee3-41154e6a4832 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:22.867328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758df9e5-145c-41f9-82a8-9c53c7eeeeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:22.868044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758df9e5-145c-41f9-82a8-9c53c7eeeeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:22.868392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758df9e5-145c-41f9-82a8-9c53c7eeeeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:22.868761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758df9e5-145c-41f9-82a8-9c53c7eeeeb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:22.869701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758df9e5-145c-41f9-82a8-9c53c7eeeeb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:22.871366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758df9e5-145c-41f9-82a8-9c53c7eeeeb6 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:22.883371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1062/2146] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:22.888162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b04da7-ef0e-4740-af74-20bed6620814 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:22.899937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b04da7-ef0e-4740-af74-20bed6620814 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:22.906293 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7fa82-e497-4d77-aeec-b3734e063b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:22.907081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7fa82-e497-4d77-aeec-b3734e063b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:22.907460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7fa82-e497-4d77-aeec-b3734e063b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:22.908229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7fa82-e497-4d77-aeec-b3734e063b25 admin admin] PATH_INFO: `/v3/projects/42628cea306d46588e875127f6f958b8/users/0b5fbd4828104cf99918ec9dcf643297/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:22.910230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7fa82-e497-4d77-aeec-b3734e063b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42628cea306d46588e875127f6f958b8, user_id=0b5fbd4828104cf99918ec9dcf643297, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:22.912912 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7fa82-e497-4d77-aeec-b3734e063b25 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:22.927815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1085/2147] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:22 2022] PUT /identity/v3/projects/42628cea306d46588e875127f6f958b8/users/0b5fbd4828104cf99918ec9dcf643297/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:22.938189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2676a-7666-446a-abff-a159450e67c2 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:22.938508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2676a-7666-446a-abff-a159450e67c2 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:22.938785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2676a-7666-446a-abff-a159450e67c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:22.975795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-11a2676a-7666-446a-abff-a159450e67c2 None None] MFA Rules not processed for user `0b5fbd4828104cf99918ec9dcf643297`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:22.977035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-11a2676a-7666-446a-abff-a159450e67c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:23.149682 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1063/2148] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:23.160835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7fa82-e497-4d77-aeec-b3734e063b25 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:23.162773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-64d7fa82-e497-4d77-aeec-b3734e063b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:23.260025 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7fa82-e497-4d77-aeec-b3734e063b25 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:23.264116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a43990-a6e3-40a3-857c-6e37b227fad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:23.264723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a43990-a6e3-40a3-857c-6e37b227fad8 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:23.265002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a43990-a6e3-40a3-857c-6e37b227fad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:23.265281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a43990-a6e3-40a3-857c-6e37b227fad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:23.269344 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f2a43990-a6e3-40a3-857c-6e37b227fad8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:23.341590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a43990-a6e3-40a3-857c-6e37b227fad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:23.343538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a43990-a6e3-40a3-857c-6e37b227fad8 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:23.367041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:23.373034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1086/2149] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:23.373383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2676a-7666-446a-abff-a159450e67c2 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:23.375001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-11a2676a-7666-446a-abff-a159450e67c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:23.550276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2676a-7666-446a-abff-a159450e67c2 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:23.583661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:23.584526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:23.584782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:23.585065 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] PATH_INFO: `/v3/projects/4e0ec516b85c4e5db232a791b9fa5134` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:23.590805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e0ec516b85c4e5db232a791b9fa5134)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:23.594921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:23.763363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0ec516b85c4e5db232a791b9fa5134', 'request_id': 'req-6b4cb527-31c0-4845-b65b-d036e406fa56'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:23.884252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0ec516b85c4e5db232a791b9fa5134', 'request_id': 'req-6b4cb527-31c0-4845-b65b-d036e406fa56'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:23.893556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0ec516b85c4e5db232a791b9fa5134', 'request_id': 'req-6b4cb527-31c0-4845-b65b-d036e406fa56'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:23.905659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] The token cache is being invalidate because project 4e0ec516b85c4e5db232a791b9fa5134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:23.905977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:23.906730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6b4cb527-31c0-4845-b65b-d036e406fa56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:23.908288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1064/2150] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:23 2022] DELETE /identity/v3/projects/4e0ec516b85c4e5db232a791b9fa5134 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:23.913736 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f9b97-bc03-4da9-b86c-ae84711fb46f None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:23.914037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f9b97-bc03-4da9-b86c-ae84711fb46f None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:23.914281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f9b97-bc03-4da9-b86c-ae84711fb46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:23.941597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-f29f9b97-bc03-4da9-b86c-ae84711fb46f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:23.942854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f29f9b97-bc03-4da9-b86c-ae84711fb46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:24.048403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1087/2151] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:24.057804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b2c90-d32b-417b-abe3-117b40dead87 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:24.058083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b2c90-d32b-417b-abe3-117b40dead87 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:24.058363 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b2c90-d32b-417b-abe3-117b40dead87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:24.091733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-4e6b2c90-d32b-417b-abe3-117b40dead87 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:24.093912 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4e6b2c90-d32b-417b-abe3-117b40dead87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:24.111503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1065/2152] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:24.112754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f9b97-bc03-4da9-b86c-ae84711fb46f None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:24.129125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f9b97-bc03-4da9-b86c-ae84711fb46f None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:24.136151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce04f9-7f2d-4583-ab79-a315735b64ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:24.136739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce04f9-7f2d-4583-ab79-a315735b64ae admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:24.137019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce04f9-7f2d-4583-ab79-a315735b64ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:24.137470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce04f9-7f2d-4583-ab79-a315735b64ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:24.138132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce04f9-7f2d-4583-ab79-a315735b64ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:24.138434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce04f9-7f2d-4583-ab79-a315735b64ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:24.140168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce04f9-7f2d-4583-ab79-a315735b64ae admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:24.154310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1088/2153] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:24:24.158944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b2c90-d32b-417b-abe3-117b40dead87 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:24.170670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b2c90-d32b-417b-abe3-117b40dead87 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:24.180274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1004802-94ec-45b1-aa29-27d788701d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:24.180274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1004802-94ec-45b1-aa29-27d788701d15 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:24.180274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1004802-94ec-45b1-aa29-27d788701d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:24.180274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1004802-94ec-45b1-aa29-27d788701d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:24.180740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1004802-94ec-45b1-aa29-27d788701d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:24.183159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1004802-94ec-45b1-aa29-27d788701d15 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:24.340873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1066/2154] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:24 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:24.341980 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce04f9-7f2d-4583-ab79-a315735b64ae admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:24.352343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce04f9-7f2d-4583-ab79-a315735b64ae admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:24.474477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8837f-d9b8-4e35-a87c-87e8900170a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:24.475427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8837f-d9b8-4e35-a87c-87e8900170a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:24.475705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8837f-d9b8-4e35-a87c-87e8900170a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:24.476058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8837f-d9b8-4e35-a87c-87e8900170a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:24.476991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8837f-d9b8-4e35-a87c-87e8900170a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:24.479111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8837f-d9b8-4e35-a87c-87e8900170a1 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:24.538844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:24.571055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1089/2155] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:24 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:24.575275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1004802-94ec-45b1-aa29-27d788701d15 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:24.585781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1004802-94ec-45b1-aa29-27d788701d15 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:24.590766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9504d-c9c5-4e9e-ac3f-cf6e5650a83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:24.591387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9504d-c9c5-4e9e-ac3f-cf6e5650a83a admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:24.591666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9504d-c9c5-4e9e-ac3f-cf6e5650a83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:24.592044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9504d-c9c5-4e9e-ac3f-cf6e5650a83a admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:24.592694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9504d-c9c5-4e9e-ac3f-cf6e5650a83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:24.594399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9504d-c9c5-4e9e-ac3f-cf6e5650a83a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:24.605632 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1067/2156] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:24.609955 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8837f-d9b8-4e35-a87c-87e8900170a1 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:24.625552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8837f-d9b8-4e35-a87c-87e8900170a1 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:24.633358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b7ea5-40b8-466a-807a-da0d2fb074bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:24.634256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7ea5-40b8-466a-807a-da0d2fb074bf admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:24.634540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7ea5-40b8-466a-807a-da0d2fb074bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:24.634816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b7ea5-40b8-466a-807a-da0d2fb074bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:24.635465 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b7ea5-40b8-466a-807a-da0d2fb074bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:24.640726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b7ea5-40b8-466a-807a-da0d2fb074bf admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:24.657831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1090/2157] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:24.661874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9504d-c9c5-4e9e-ac3f-cf6e5650a83a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:24.672738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9504d-c9c5-4e9e-ac3f-cf6e5650a83a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:24.679222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbe18c-db0e-4600-8a5e-c0a489f5d857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:24.679919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbe18c-db0e-4600-8a5e-c0a489f5d857 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:24.680206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbe18c-db0e-4600-8a5e-c0a489f5d857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:24.680487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbe18c-db0e-4600-8a5e-c0a489f5d857 admin admin] PATH_INFO: `/v3/projects/98dc17238e154072b41ce1de1b8c22a5/users/f1d9452012f146758f81e78a104e7aba/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:24.707342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbe18c-db0e-4600-8a5e-c0a489f5d857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98dc17238e154072b41ce1de1b8c22a5, user_id=f1d9452012f146758f81e78a104e7aba, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:24.711131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbe18c-db0e-4600-8a5e-c0a489f5d857 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:24.802692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1068/2158] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:24 2022] PUT /identity/v3/projects/98dc17238e154072b41ce1de1b8c22a5/users/f1d9452012f146758f81e78a104e7aba/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:24.809462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52514ec-d9bc-4b86-ac52-812f6bbc4d56 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:24.809811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52514ec-d9bc-4b86-ac52-812f6bbc4d56 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:24.810097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52514ec-d9bc-4b86-ac52-812f6bbc4d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:24.842491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-c52514ec-d9bc-4b86-ac52-812f6bbc4d56 None None] MFA Rules not processed for user `f1d9452012f146758f81e78a104e7aba`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:24.843719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c52514ec-d9bc-4b86-ac52-812f6bbc4d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:24.913648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1091/2159] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:24.926467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbe18c-db0e-4600-8a5e-c0a489f5d857 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:24.928349 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8ebbe18c-db0e-4600-8a5e-c0a489f5d857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:25.008640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbe18c-db0e-4600-8a5e-c0a489f5d857 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:25.012531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76027c3c-0bff-4ba8-b9c1-90260fef2948 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5d7cf7534941cb857a4ab5867d978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:25.013183 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76027c3c-0bff-4ba8-b9c1-90260fef2948 service glance] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:25.013469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76027c3c-0bff-4ba8-b9c1-90260fef2948 service glance] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:25.013871 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76027c3c-0bff-4ba8-b9c1-90260fef2948 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:25.022788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76027c3c-0bff-4ba8-b9c1-90260fef2948 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:25.024262 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76027c3c-0bff-4ba8-b9c1-90260fef2948 service glance] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:25.033248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1069/2160] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:25.981296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:26.546669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:26.895685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:27.101163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:27.226736 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:27.278795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:27.392484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:27.421490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:27.518177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:27.541291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:27.595621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:27.624208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:27.747152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:31.240267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52514ec-d9bc-4b86-ac52-812f6bbc4d56 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:31.241698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c52514ec-d9bc-4b86-ac52-812f6bbc4d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:31.311206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52514ec-d9bc-4b86-ac52-812f6bbc4d56 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:31.315383 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b40c89-038e-47f2-b299-bd2e8973c163 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:31.316042 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b40c89-038e-47f2-b299-bd2e8973c163 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:31.316422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b40c89-038e-47f2-b299-bd2e8973c163 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:31.317261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b40c89-038e-47f2-b299-bd2e8973c163 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:31.319117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e8b40c89-038e-47f2-b299-bd2e8973c163 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:31.395509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b40c89-038e-47f2-b299-bd2e8973c163 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:31.397642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b40c89-038e-47f2-b299-bd2e8973c163 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:31.420258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1092/2161] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:33.466398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76027c3c-0bff-4ba8-b9c1-90260fef2948 service glance] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:33.477130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76027c3c-0bff-4ba8-b9c1-90260fef2948 service glance] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:33.481023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89a0e1-80ae-47f1-873f-582a6a3e79da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:33.481683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89a0e1-80ae-47f1-873f-582a6a3e79da service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:33.482105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89a0e1-80ae-47f1-873f-582a6a3e79da service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:33.482266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89a0e1-80ae-47f1-873f-582a6a3e79da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:33.484710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ad89a0e1-80ae-47f1-873f-582a6a3e79da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:33.562256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89a0e1-80ae-47f1-873f-582a6a3e79da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:33.563709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89a0e1-80ae-47f1-873f-582a6a3e79da service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:33.575515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1070/2162] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:36.405338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b40c89-038e-47f2-b299-bd2e8973c163 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:36.406730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e8b40c89-038e-47f2-b299-bd2e8973c163 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:36.478552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b40c89-038e-47f2-b299-bd2e8973c163 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:36.487103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24132202-61e1-4cc9-8daa-8253752d6f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:36.487103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132202-61e1-4cc9-8daa-8253752d6f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:36.487103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132202-61e1-4cc9-8daa-8253752d6f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:36.487103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24132202-61e1-4cc9-8daa-8253752d6f43 admin admin] PATH_INFO: `/v3/users/aab79224668f4beeb57a4314fdadfb46` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:36.487460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24132202-61e1-4cc9-8daa-8253752d6f43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab79224668f4beeb57a4314fdadfb46)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:36.487672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24132202-61e1-4cc9-8daa-8253752d6f43 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:36.690947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-24132202-61e1-4cc9-8daa-8253752d6f43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab79224668f4beeb57a4314fdadfb46'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:36.694255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:36.730961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89a0e1-80ae-47f1-873f-582a6a3e79da service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:36.732687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ad89a0e1-80ae-47f1-873f-582a6a3e79da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:36.746540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89a0e1-80ae-47f1-873f-582a6a3e79da service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:36.751357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:36.752119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:36.752401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:36.752681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] PATH_INFO: `/v3/users/c0019e40c05c4b8a8387584fff6e12ea` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:36.754354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0019e40c05c4b8a8387584fff6e12ea)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:36.756415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:36.760483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-24132202-61e1-4cc9-8daa-8253752d6f43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab79224668f4beeb57a4314fdadfb46'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:36.761384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-24132202-61e1-4cc9-8daa-8253752d6f43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab79224668f4beeb57a4314fdadfb46'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:36.763932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-24132202-61e1-4cc9-8daa-8253752d6f43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab79224668f4beeb57a4314fdadfb46'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:36.772016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-24132202-61e1-4cc9-8daa-8253752d6f43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab79224668f4beeb57a4314fdadfb46'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:36.792426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1093/2163] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:36 2022] DELETE /identity/v3/users/aab79224668f4beeb57a4314fdadfb46 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:36.797039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f79f91-4e65-4aeb-970d-a267f002577e None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:36.797309 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f79f91-4e65-4aeb-970d-a267f002577e None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:36.797810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f79f91-4e65-4aeb-970d-a267f002577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:36.816703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-00f79f91-4e65-4aeb-970d-a267f002577e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:36.817745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-00f79f91-4e65-4aeb-970d-a267f002577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:36.890194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1094/2164] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:36.900479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f79f91-4e65-4aeb-970d-a267f002577e None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:36.901878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-00f79f91-4e65-4aeb-970d-a267f002577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:36.961621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f79f91-4e65-4aeb-970d-a267f002577e None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:36.966190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4759405-cf21-49a9-ab39-02ef354d1e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:36.966542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0019e40c05c4b8a8387584fff6e12ea'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:36.966801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4759405-cf21-49a9-ab39-02ef354d1e57 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:36.967078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4759405-cf21-49a9-ab39-02ef354d1e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:36.967348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4759405-cf21-49a9-ab39-02ef354d1e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:36.976953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4759405-cf21-49a9-ab39-02ef354d1e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:36.978523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4759405-cf21-49a9-ab39-02ef354d1e57 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:36.988529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1095/2165] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:24:37.041424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0019e40c05c4b8a8387584fff6e12ea'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:37.041424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0019e40c05c4b8a8387584fff6e12ea'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:37.041424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0019e40c05c4b8a8387584fff6e12ea'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:37.043284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0019e40c05c4b8a8387584fff6e12ea'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:37.062315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1071/2166] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:36 2022] DELETE /identity/v3/users/c0019e40c05c4b8a8387584fff6e12ea => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:37.067404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a50037-8000-4a2f-aafb-a7bad2e0a69b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:37.067685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a50037-8000-4a2f-aafb-a7bad2e0a69b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:37.067979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a50037-8000-4a2f-aafb-a7bad2e0a69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:37.092553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-f6a50037-8000-4a2f-aafb-a7bad2e0a69b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:37.093972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f6a50037-8000-4a2f-aafb-a7bad2e0a69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:37.155821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1096/2167] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:37.165914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:37.167457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:37.233016 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e39396a-a299-460b-be6a-2a68cd0567cf admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:37.240336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182b15e-5a6e-4588-b001-a23f8b07cffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:37.241214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182b15e-5a6e-4588-b001-a23f8b07cffa service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:37.241862 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182b15e-5a6e-4588-b001-a23f8b07cffa service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:37.242308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182b15e-5a6e-4588-b001-a23f8b07cffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:37.251499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:37.260179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182b15e-5a6e-4588-b001-a23f8b07cffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:37.261258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182b15e-5a6e-4588-b001-a23f8b07cffa service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:37.274169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1072/2168] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:37.501203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a50037-8000-4a2f-aafb-a7bad2e0a69b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:37.501203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f6a50037-8000-4a2f-aafb-a7bad2e0a69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:37.508486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a50037-8000-4a2f-aafb-a7bad2e0a69b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:37.514621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:37.515872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:37.516234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:37.516586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] PATH_INFO: `/v3/projects/8235aa8e9b0f46e9837bae37e07afb68` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:37.517834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8235aa8e9b0f46e9837bae37e07afb68)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:37.520192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:37.529838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:37.676793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8235aa8e9b0f46e9837bae37e07afb68', 'request_id': 'req-af0ecc4a-c41a-4913-a98f-41d4d91775b5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:37.682924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8235aa8e9b0f46e9837bae37e07afb68', 'request_id': 'req-af0ecc4a-c41a-4913-a98f-41d4d91775b5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:37.766260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8235aa8e9b0f46e9837bae37e07afb68', 'request_id': 'req-af0ecc4a-c41a-4913-a98f-41d4d91775b5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:37.781284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] The token cache is being invalidate because project 8235aa8e9b0f46e9837bae37e07afb68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:37.781600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:37.782249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-af0ecc4a-c41a-4913-a98f-41d4d91775b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:37.785509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1097/2169] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:37 2022] DELETE /identity/v3/projects/8235aa8e9b0f46e9837bae37e07afb68 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:37.795411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1337d5d-14d9-4a4d-bf0c-b8da336e6f1a None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:37.795943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1337d5d-14d9-4a4d-bf0c-b8da336e6f1a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:37.796039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1337d5d-14d9-4a4d-bf0c-b8da336e6f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:37.818148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-f1337d5d-14d9-4a4d-bf0c-b8da336e6f1a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:37.819699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f1337d5d-14d9-4a4d-bf0c-b8da336e6f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:37.879699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1073/2170] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:37.895605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c0a0b-ed4f-46c4-8220-1ce8069d9d38 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:37.895929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c0a0b-ed4f-46c4-8220-1ce8069d9d38 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:37.896324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c0a0b-ed4f-46c4-8220-1ce8069d9d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:37.917309 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-f56c0a0b-ed4f-46c4-8220-1ce8069d9d38 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:37.918503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f56c0a0b-ed4f-46c4-8220-1ce8069d9d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:37.927580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1098/2171] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:37.933109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1337d5d-14d9-4a4d-bf0c-b8da336e6f1a None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:37.936936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c0a0b-ed4f-46c4-8220-1ce8069d9d38 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:37.946627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f56c0a0b-ed4f-46c4-8220-1ce8069d9d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:37.946727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1337d5d-14d9-4a4d-bf0c-b8da336e6f1a None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:37.953503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe65f9-ac2e-44ce-8c96-11c5db89b377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:37.954130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe65f9-ac2e-44ce-8c96-11c5db89b377 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:37.954391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe65f9-ac2e-44ce-8c96-11c5db89b377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:37.954667 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe65f9-ac2e-44ce-8c96-11c5db89b377 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:37.957307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe65f9-ac2e-44ce-8c96-11c5db89b377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:37.957307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe65f9-ac2e-44ce-8c96-11c5db89b377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:37.957608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c0a0b-ed4f-46c4-8220-1ce8069d9d38 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:37.957868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe65f9-ac2e-44ce-8c96-11c5db89b377 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:37.967375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:37.968441 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:37.968819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:37.969274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] PATH_INFO: `/v3/projects/f14c1634c5c44bcb966fc2817a94a6f4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:37.975429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1074/2172] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:24:37.977127 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe65f9-ac2e-44ce-8c96-11c5db89b377 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:37.980859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f14c1634c5c44bcb966fc2817a94a6f4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:37.983449 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:37.988000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe65f9-ac2e-44ce-8c96-11c5db89b377 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:37.993073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098ad83-8355-421b-b7bc-2f04e2d14719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:37.993686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098ad83-8355-421b-b7bc-2f04e2d14719 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:37.994285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098ad83-8355-421b-b7bc-2f04e2d14719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:37.994412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b098ad83-8355-421b-b7bc-2f04e2d14719 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:37.994988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098ad83-8355-421b-b7bc-2f04e2d14719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:37.996929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b098ad83-8355-421b-b7bc-2f04e2d14719 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:38.098271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f14c1634c5c44bcb966fc2817a94a6f4', 'request_id': 'req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:38.103689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f14c1634c5c44bcb966fc2817a94a6f4', 'request_id': 'req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:38.115998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1075/2173] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:37 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:38.120268 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098ad83-8355-421b-b7bc-2f04e2d14719 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:38.190032 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b098ad83-8355-421b-b7bc-2f04e2d14719 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:38.196047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b158c46-3655-4934-a40e-f17811c50c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:38.196779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b158c46-3655-4934-a40e-f17811c50c36 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:38.197180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b158c46-3655-4934-a40e-f17811c50c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:38.197592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b158c46-3655-4934-a40e-f17811c50c36 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:38.198355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b158c46-3655-4934-a40e-f17811c50c36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:38.200408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b158c46-3655-4934-a40e-f17811c50c36 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:38.206302 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f14c1634c5c44bcb966fc2817a94a6f4', 'request_id': 'req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:38.228535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] The token cache is being invalidate because project f14c1634c5c44bcb966fc2817a94a6f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:38.229041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:38.230015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:38.235448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1099/2174] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:37 2022] DELETE /identity/v3/projects/f14c1634c5c44bcb966fc2817a94a6f4 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:38.236702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:38.238682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:38.293238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1076/2175] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:38 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:38.298166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b158c46-3655-4934-a40e-f17811c50c36 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:38.299934 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3b158c46-3655-4934-a40e-f17811c50c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:38.312685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b158c46-3655-4934-a40e-f17811c50c36 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:38.322624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df351ec3-d76c-41b7-8969-e852af8feb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:38.322624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df351ec3-d76c-41b7-8969-e852af8feb43 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:38.322624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df351ec3-d76c-41b7-8969-e852af8feb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:38.322624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df351ec3-d76c-41b7-8969-e852af8feb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:38.323023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df351ec3-d76c-41b7-8969-e852af8feb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:38.323121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331aa3f-7f04-4a9b-b5f6-4f2d77c5a101 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:38.323213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df351ec3-d76c-41b7-8969-e852af8feb43 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:38.326399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:38.326963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:38.327342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:38.327687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] PATH_INFO: `/v3/users/6b68fec240c348da85f1a032846e876d` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:38.328938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b68fec240c348da85f1a032846e876d)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:38.330845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:38.333448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1077/2176] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:24:38.337635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df351ec3-d76c-41b7-8969-e852af8feb43 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:38.347000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:38.347224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df351ec3-d76c-41b7-8969-e852af8feb43 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:38.351992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157ad0c-a61b-4e4e-84f2-6093071051ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:38.352579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157ad0c-a61b-4e4e-84f2-6093071051ee admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:38.352858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157ad0c-a61b-4e4e-84f2-6093071051ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:38.353130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d157ad0c-a61b-4e4e-84f2-6093071051ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:38.353763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157ad0c-a61b-4e4e-84f2-6093071051ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:38.355336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d157ad0c-a61b-4e4e-84f2-6093071051ee admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:38.367025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1078/2177] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:24:38.370670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157ad0c-a61b-4e4e-84f2-6093071051ee admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:38.381246 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d157ad0c-a61b-4e4e-84f2-6093071051ee admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:38.385959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdc630-2a85-4720-9979-eea0f6ae4cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:38.386597 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdc630-2a85-4720-9979-eea0f6ae4cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:38.386960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdc630-2a85-4720-9979-eea0f6ae4cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:38.387238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdc630-2a85-4720-9979-eea0f6ae4cdb admin admin] PATH_INFO: `/v3/projects/3811dcff000c4821bd1d1a2715d1fd21/users/afe1f2aeb4eb4f31a91e031d3a0f3a9f/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:38.403943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdc630-2a85-4720-9979-eea0f6ae4cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=3811dcff000c4821bd1d1a2715d1fd21, user_id=afe1f2aeb4eb4f31a91e031d3a0f3a9f, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:38.406748 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdc630-2a85-4720-9979-eea0f6ae4cdb admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:38.511339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1079/2178] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:38 2022] PUT /identity/v3/projects/3811dcff000c4821bd1d1a2715d1fd21/users/afe1f2aeb4eb4f31a91e031d3a0f3a9f/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:38.522985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d8877-93e5-4be7-954f-cf86667ca6b4 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:38.523270 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d8877-93e5-4be7-954f-cf86667ca6b4 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:38.523551 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d8877-93e5-4be7-954f-cf86667ca6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:38.531478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:38.554771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ed6d8877-93e5-4be7-954f-cf86667ca6b4 None None] MFA Rules not processed for user `afe1f2aeb4eb4f31a91e031d3a0f3a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:38.556178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ed6d8877-93e5-4be7-954f-cf86667ca6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:38.558138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b68fec240c348da85f1a032846e876d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:38.619809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1080/2179] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:38.629528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b68fec240c348da85f1a032846e876d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:38.630146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b68fec240c348da85f1a032846e876d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:38.630744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b68fec240c348da85f1a032846e876d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:38.632044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d8877-93e5-4be7-954f-cf86667ca6b4 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:38.633515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ed6d8877-93e5-4be7-954f-cf86667ca6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:38.640120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b68fec240c348da85f1a032846e876d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:38.660160 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1100/2180] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:38 2022] DELETE /identity/v3/users/6b68fec240c348da85f1a032846e876d => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:38.696956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d8877-93e5-4be7-954f-cf86667ca6b4 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:38.749147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf7105-a550-4512-ab9d-4ec25d297620 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5d7cf7534941cb857a4ab5867d978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:38.749819 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf7105-a550-4512-ab9d-4ec25d297620 service glance] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:38.750142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf7105-a550-4512-ab9d-4ec25d297620 service glance] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:38.750485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf7105-a550-4512-ab9d-4ec25d297620 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:38.752721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-fedf7105-a550-4512-ab9d-4ec25d297620 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:38.813556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf7105-a550-4512-ab9d-4ec25d297620 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:38.814933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedf7105-a550-4512-ab9d-4ec25d297620 service glance] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:38.824273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1081/2181] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:24:39.111012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:39.114623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:39.199748 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a5a05b-1962-4768-87af-1df1379feb3e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:39.205030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:39.205683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:39.205959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:39.206238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] PATH_INFO: `/v3/projects/3271c231153949b4a23a2d0a6f92e3c1` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:39.208023 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3271c231153949b4a23a2d0a6f92e3c1)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:39.210090 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:39.317046 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3271c231153949b4a23a2d0a6f92e3c1', 'request_id': 'req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:39.325846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3271c231153949b4a23a2d0a6f92e3c1', 'request_id': 'req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:39.412255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3271c231153949b4a23a2d0a6f92e3c1', 'request_id': 'req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:39.424828 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] The token cache is being invalidate because project 3271c231153949b4a23a2d0a6f92e3c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:39.425197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:39.425994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-cd4d8dd8-ecaa-4553-8afa-26e95afc0d98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:39.427498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1101/2182] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:39 2022] DELETE /identity/v3/projects/3271c231153949b4a23a2d0a6f92e3c1 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:39.432581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ed3ed-2116-4568-980e-7217b05e81d4 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:39.432875 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ed3ed-2116-4568-980e-7217b05e81d4 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:39.433184 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ed3ed-2116-4568-980e-7217b05e81d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:39.442976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:39.456043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-086ed3ed-2116-4568-980e-7217b05e81d4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:39.458274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-086ed3ed-2116-4568-980e-7217b05e81d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:39.536874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1082/2183] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:39.547490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb764e70-8c40-4ea4-b241-6d9b1d2150af None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:39.548442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb764e70-8c40-4ea4-b241-6d9b1d2150af None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:39.548746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb764e70-8c40-4ea4-b241-6d9b1d2150af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:39.575333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-cb764e70-8c40-4ea4-b241-6d9b1d2150af None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:39.576526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-cb764e70-8c40-4ea4-b241-6d9b1d2150af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:39.585779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ed3ed-2116-4568-980e-7217b05e81d4 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:39.587252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-086ed3ed-2116-4568-980e-7217b05e81d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:39.589263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1102/2184] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:39.592971 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb764e70-8c40-4ea4-b241-6d9b1d2150af None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:39.599791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ed3ed-2116-4568-980e-7217b05e81d4 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:39.606311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b7c17-055c-4bee-a91b-d471eee96791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:39.606874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb764e70-8c40-4ea4-b241-6d9b1d2150af None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:39.607067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b7c17-055c-4bee-a91b-d471eee96791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:39.607288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b7c17-055c-4bee-a91b-d471eee96791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:39.607619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b7c17-055c-4bee-a91b-d471eee96791 admin admin] PATH_INFO: `/v3/users/ca6da8e3af1b43da992a378388200dff` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:39.608890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b7c17-055c-4bee-a91b-d471eee96791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca6da8e3af1b43da992a378388200dff)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:39.614379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b7c17-055c-4bee-a91b-d471eee96791 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:39.631984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c01b9b-4346-411f-9cd2-fb23dc7789ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:39.631984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01b9b-4346-411f-9cd2-fb23dc7789ab admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:39.631984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01b9b-4346-411f-9cd2-fb23dc7789ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:39.631984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01b9b-4346-411f-9cd2-fb23dc7789ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:39.632371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c01b9b-4346-411f-9cd2-fb23dc7789ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:39.632371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c01b9b-4346-411f-9cd2-fb23dc7789ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:39.633425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c01b9b-4346-411f-9cd2-fb23dc7789ab admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:39.649639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:39.653778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1103/2185] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:24:39.663978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c01b9b-4346-411f-9cd2-fb23dc7789ab admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:39.676379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c01b9b-4346-411f-9cd2-fb23dc7789ab admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:39.682073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5068d-df4b-4198-8978-dd474ae254fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:39.682606 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5068d-df4b-4198-8978-dd474ae254fb admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:39.682885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5068d-df4b-4198-8978-dd474ae254fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:39.683158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5068d-df4b-4198-8978-dd474ae254fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:39.683878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5068d-df4b-4198-8978-dd474ae254fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:39.686555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5068d-df4b-4198-8978-dd474ae254fb admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:39.794117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1104/2186] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:39 2022] POST /identity/v3/projects => generated 363 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:39.798454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5068d-df4b-4198-8978-dd474ae254fb admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:39.820631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6e0b7c17-055c-4bee-a91b-d471eee96791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6da8e3af1b43da992a378388200dff'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:39.873405 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5068d-df4b-4198-8978-dd474ae254fb admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:39.879994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561d20a-61b4-4d10-88de-8740aa31e05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:39.880592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561d20a-61b4-4d10-88de-8740aa31e05a admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:39.880872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561d20a-61b4-4d10-88de-8740aa31e05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:39.881147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561d20a-61b4-4d10-88de-8740aa31e05a admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:39.881783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561d20a-61b4-4d10-88de-8740aa31e05a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:39.883707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d561d20a-61b4-4d10-88de-8740aa31e05a admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:39.892196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6e0b7c17-055c-4bee-a91b-d471eee96791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6da8e3af1b43da992a378388200dff'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:39.892847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6e0b7c17-055c-4bee-a91b-d471eee96791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6da8e3af1b43da992a378388200dff'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:39.893453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6e0b7c17-055c-4bee-a91b-d471eee96791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6da8e3af1b43da992a378388200dff'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:39.909404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-6e0b7c17-055c-4bee-a91b-d471eee96791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6da8e3af1b43da992a378388200dff'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:39.928999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1083/2187] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:39 2022] DELETE /identity/v3/users/ca6da8e3af1b43da992a378388200dff => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:39.934969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d026d1-01ad-4bf6-89e6-8655711605d4 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:39.935261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d026d1-01ad-4bf6-89e6-8655711605d4 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:39.935544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d026d1-01ad-4bf6-89e6-8655711605d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:39.956698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-85d026d1-01ad-4bf6-89e6-8655711605d4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:39.957907 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-85d026d1-01ad-4bf6-89e6-8655711605d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:40.010207 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1105/2188] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:39 2022] POST /identity/v3/users => generated 282 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:40.014761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561d20a-61b4-4d10-88de-8740aa31e05a admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:40.016598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d561d20a-61b4-4d10-88de-8740aa31e05a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:40.022592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1084/2189] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:40.031180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561d20a-61b4-4d10-88de-8740aa31e05a admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:40.032921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d026d1-01ad-4bf6-89e6-8655711605d4 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:40.034303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-85d026d1-01ad-4bf6-89e6-8655711605d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:40.036241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac138fe-f61a-4dc8-854b-5190a9bb8848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:40.036847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac138fe-f61a-4dc8-854b-5190a9bb8848 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:40.037131 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac138fe-f61a-4dc8-854b-5190a9bb8848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:40.037408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac138fe-f61a-4dc8-854b-5190a9bb8848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:40.038053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac138fe-f61a-4dc8-854b-5190a9bb8848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:40.039678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac138fe-f61a-4dc8-854b-5190a9bb8848 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:40.050946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1106/2190] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:24:40.053633 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:40.054876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac138fe-f61a-4dc8-854b-5190a9bb8848 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:40.065649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac138fe-f61a-4dc8-854b-5190a9bb8848 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:40.071670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57096-e3e8-4c10-a958-ba03b273e6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:40.072306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57096-e3e8-4c10-a958-ba03b273e6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:40.072588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57096-e3e8-4c10-a958-ba03b273e6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:40.072864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57096-e3e8-4c10-a958-ba03b273e6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:40.073507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57096-e3e8-4c10-a958-ba03b273e6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:40.075113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57096-e3e8-4c10-a958-ba03b273e6d0 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:40.087887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1107/2191] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:24:40.091752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57096-e3e8-4c10-a958-ba03b273e6d0 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:40.101142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d026d1-01ad-4bf6-89e6-8655711605d4 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:40.103336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57096-e3e8-4c10-a958-ba03b273e6d0 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:40.104870 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77cb7d-d051-4841-bd7a-c536cbfdbeed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:40.105479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77cb7d-d051-4841-bd7a-c536cbfdbeed service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:40.105758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77cb7d-d051-4841-bd7a-c536cbfdbeed service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:40.107176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77cb7d-d051-4841-bd7a-c536cbfdbeed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:40.108739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557f8cb-3356-4d48-a96a-b560e2289580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:40.109387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557f8cb-3356-4d48-a96a-b560e2289580 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:40.109684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557f8cb-3356-4d48-a96a-b560e2289580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:40.109943 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557f8cb-3356-4d48-a96a-b560e2289580 admin admin] PATH_INFO: `/v3/projects/5d62a93c2cfb4c3681e08d4e9457f113/users/779987ce97764b18a7207c2523365c23/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:40.117771 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77cb7d-d051-4841-bd7a-c536cbfdbeed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:40.119278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77cb7d-d051-4841-bd7a-c536cbfdbeed service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:40.127263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557f8cb-3356-4d48-a96a-b560e2289580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d62a93c2cfb4c3681e08d4e9457f113, user_id=779987ce97764b18a7207c2523365c23, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:40.128126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557f8cb-3356-4d48-a96a-b560e2289580 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:40.129816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1085/2192] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:24:40.201235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1108/2193] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:40 2022] PUT /identity/v3/projects/5d62a93c2cfb4c3681e08d4e9457f113/users/779987ce97764b18a7207c2523365c23/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:40.211362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23b843-0097-499a-a49f-163e654d2551 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:40.211828 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23b843-0097-499a-a49f-163e654d2551 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:40.212194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db23b843-0097-499a-a49f-163e654d2551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:40.246531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-db23b843-0097-499a-a49f-163e654d2551 None None] MFA Rules not processed for user `779987ce97764b18a7207c2523365c23`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:40.247972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-db23b843-0097-499a-a49f-163e654d2551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:40.322880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1086/2194] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:40.324060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557f8cb-3356-4d48-a96a-b560e2289580 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:40.408034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557f8cb-3356-4d48-a96a-b560e2289580 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:40.414983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60eced-9262-474e-9dd1-82db730e548e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:40.415720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60eced-9262-474e-9dd1-82db730e548e admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:40.416074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60eced-9262-474e-9dd1-82db730e548e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:40.416406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60eced-9262-474e-9dd1-82db730e548e admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:40.417276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60eced-9262-474e-9dd1-82db730e548e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:40.419882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60eced-9262-474e-9dd1-82db730e548e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:40.540556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1109/2195] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:40 2022] POST /identity/v3/projects => generated 365 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:40.545283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23b843-0097-499a-a49f-163e654d2551 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:40.556040 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db23b843-0097-499a-a49f-163e654d2551 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:40.646113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b276494-cd44-434f-b66f-8c69f82ff8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:40.646940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b276494-cd44-434f-b66f-8c69f82ff8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:40.647939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b276494-cd44-434f-b66f-8c69f82ff8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:40.648290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b276494-cd44-434f-b66f-8c69f82ff8a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:40.649028 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b276494-cd44-434f-b66f-8c69f82ff8a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:40.651600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b276494-cd44-434f-b66f-8c69f82ff8a7 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:40.763747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1087/2196] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:40 2022] POST /identity/v3/users => generated 289 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:40.767879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60eced-9262-474e-9dd1-82db730e548e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:40.778412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60eced-9262-474e-9dd1-82db730e548e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:40.784207 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d9f4f-5e74-4f67-814b-ec6d8297714c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:40.785124 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d9f4f-5e74-4f67-814b-ec6d8297714c admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:40.785407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d9f4f-5e74-4f67-814b-ec6d8297714c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:40.785699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d9f4f-5e74-4f67-814b-ec6d8297714c admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:40.786346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d9f4f-5e74-4f67-814b-ec6d8297714c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:40.788368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d9f4f-5e74-4f67-814b-ec6d8297714c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:40.790147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:40.804104 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1110/2197] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:40.808880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b276494-cd44-434f-b66f-8c69f82ff8a7 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:40.822417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b276494-cd44-434f-b66f-8c69f82ff8a7 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:40.832452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012b497-89ad-45e2-89aa-b6e4742f4f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:40.833458 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012b497-89ad-45e2-89aa-b6e4742f4f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:40.833808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012b497-89ad-45e2-89aa-b6e4742f4f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:40.834330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012b497-89ad-45e2-89aa-b6e4742f4f7b admin admin] PATH_INFO: `/v3/projects/8cd6cebf3cc94c38ad35bbb1bd1d9469/users/dc4eed6e32e1401db4f0f381f6eb602d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:40.851960 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d9f4f-5e74-4f67-814b-ec6d8297714c admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:40.853288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-845d9f4f-5e74-4f67-814b-ec6d8297714c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:40.855703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012b497-89ad-45e2-89aa-b6e4742f4f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd6cebf3cc94c38ad35bbb1bd1d9469, user_id=dc4eed6e32e1401db4f0f381f6eb602d, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:40.858499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9012b497-89ad-45e2-89aa-b6e4742f4f7b admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:40.868750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d9f4f-5e74-4f67-814b-ec6d8297714c admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:40.874555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:40.875392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:40.875793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:40.876236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] PATH_INFO: `/v3/projects/4aeda7b6b8524f21ab966ab08f77a2d7` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:40.877818 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4aeda7b6b8524f21ab966ab08f77a2d7)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:40.880482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:40.969337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1088/2198] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:40 2022] PUT /identity/v3/projects/8cd6cebf3cc94c38ad35bbb1bd1d9469/users/dc4eed6e32e1401db4f0f381f6eb602d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:40.970377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012b497-89ad-45e2-89aa-b6e4742f4f7b admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:40.985318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:41.019207 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4aeda7b6b8524f21ab966ab08f77a2d7', 'request_id': 'req-6328c30c-7460-4469-a0cd-ca82fcc1be5c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:41.025706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4aeda7b6b8524f21ab966ab08f77a2d7', 'request_id': 'req-6328c30c-7460-4469-a0cd-ca82fcc1be5c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:41.045182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012b497-89ad-45e2-89aa-b6e4742f4f7b admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:41.103424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30411f5-2290-47f3-96a2-324892b20460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:41.104065 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30411f5-2290-47f3-96a2-324892b20460 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:41.104347 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30411f5-2290-47f3-96a2-324892b20460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:41.104625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30411f5-2290-47f3-96a2-324892b20460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:41.105262 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30411f5-2290-47f3-96a2-324892b20460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:41.106924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30411f5-2290-47f3-96a2-324892b20460 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:41.120205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1089/2199] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:41.121789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30411f5-2290-47f3-96a2-324892b20460 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:41.130417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4aeda7b6b8524f21ab966ab08f77a2d7', 'request_id': 'req-6328c30c-7460-4469-a0cd-ca82fcc1be5c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:41.145333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] The token cache is being invalidate because project 4aeda7b6b8524f21ab966ab08f77a2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:41.145561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:41.146238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:41.146352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30411f5-2290-47f3-96a2-324892b20460 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:41.148141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1111/2200] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:40 2022] DELETE /identity/v3/projects/4aeda7b6b8524f21ab966ab08f77a2d7 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:41.151921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:41.153253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:41.160301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658bb21-0eed-47da-bc61-9dae21dbad5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:41.160301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658bb21-0eed-47da-bc61-9dae21dbad5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:41.160301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658bb21-0eed-47da-bc61-9dae21dbad5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:41.160301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658bb21-0eed-47da-bc61-9dae21dbad5d admin admin] PATH_INFO: `/v3/projects/8cd6cebf3cc94c38ad35bbb1bd1d9469/users/dc4eed6e32e1401db4f0f381f6eb602d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:41.160605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658bb21-0eed-47da-bc61-9dae21dbad5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cd6cebf3cc94c38ad35bbb1bd1d9469, user_id=dc4eed6e32e1401db4f0f381f6eb602d, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:41.161839 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658bb21-0eed-47da-bc61-9dae21dbad5d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:41.172882 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328c30c-7460-4469-a0cd-ca82fcc1be5c admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:41.178336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1090/2201] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:41 2022] PUT /identity/v3/projects/8cd6cebf3cc94c38ad35bbb1bd1d9469/users/dc4eed6e32e1401db4f0f381f6eb602d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:41.188495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29231252-362f-4ff7-8079-d5016e5ba093 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:41.188800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29231252-362f-4ff7-8079-d5016e5ba093 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:41.189084 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29231252-362f-4ff7-8079-d5016e5ba093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:41.225311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-29231252-362f-4ff7-8079-d5016e5ba093 None None] MFA Rules not processed for user `dc4eed6e32e1401db4f0f381f6eb602d`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:41.228473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-29231252-362f-4ff7-8079-d5016e5ba093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:41.252719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:41.253693 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:41.254005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:41.258922 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] PATH_INFO: `/v3/users/52a6f19e905344fcabc3e7e5084ca9a5` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:41.261672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a6f19e905344fcabc3e7e5084ca9a5)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:41.263644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:41.343810 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1091/2202] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:41.355316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29231252-362f-4ff7-8079-d5016e5ba093 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:41.356737 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-29231252-362f-4ff7-8079-d5016e5ba093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:41.392166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:41.417849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29231252-362f-4ff7-8079-d5016e5ba093 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:41.422464 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead6d55-42f0-4e50-955f-bcf2bf92df11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:41.433224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead6d55-42f0-4e50-955f-bcf2bf92df11 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:41.433224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead6d55-42f0-4e50-955f-bcf2bf92df11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:41.433224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead6d55-42f0-4e50-955f-bcf2bf92df11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:41.438076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead6d55-42f0-4e50-955f-bcf2bf92df11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:41.439558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead6d55-42f0-4e50-955f-bcf2bf92df11 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:41.450840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1092/2203] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:24:41.473923 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a6f19e905344fcabc3e7e5084ca9a5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:41.568175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a6f19e905344fcabc3e7e5084ca9a5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:41.569648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a6f19e905344fcabc3e7e5084ca9a5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:41.576321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a6f19e905344fcabc3e7e5084ca9a5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:41.595701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646c93-d64f-4764-9d9e-2de80670d62e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:41.596027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646c93-d64f-4764-9d9e-2de80670d62e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:41.596302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646c93-d64f-4764-9d9e-2de80670d62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:41.598167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a6f19e905344fcabc3e7e5084ca9a5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:41.611853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:41.616689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1112/2204] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:41 2022] DELETE /identity/v3/users/52a6f19e905344fcabc3e7e5084ca9a5 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:41.620422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ac646c93-d64f-4764-9d9e-2de80670d62e None None] MFA Rules not processed for user `4bd826253c6b4b5096bee7ffaac04d18`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:41.621669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ac646c93-d64f-4764-9d9e-2de80670d62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:41.690578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:41.692303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1093/2205] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:41.701356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:41.782617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad664fb0-f4e3-4ac9-aff1-51ace2a78048 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:41.787177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a075b6c-1946-41cb-be05-f16521fee0f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:41.787824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a075b6c-1946-41cb-be05-f16521fee0f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:41.788109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a075b6c-1946-41cb-be05-f16521fee0f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:41.788448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a075b6c-1946-41cb-be05-f16521fee0f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:41.790184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7a075b6c-1946-41cb-be05-f16521fee0f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:41.886473 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a075b6c-1946-41cb-be05-f16521fee0f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:41.888060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a075b6c-1946-41cb-be05-f16521fee0f6 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:41.898419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1113/2206] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:24:42.066549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac646c93-d64f-4764-9d9e-2de80670d62e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:42.069067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ac646c93-d64f-4764-9d9e-2de80670d62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:42.153113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac646c93-d64f-4764-9d9e-2de80670d62e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:42.157685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:42.158320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:42.158601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:42.158876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] PATH_INFO: `/v3/projects/bff08c77760945729ee8517d6de31ce5` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:42.160076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff08c77760945729ee8517d6de31ce5)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:42.161955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:42.281243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff08c77760945729ee8517d6de31ce5', 'request_id': 'req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:42.345642 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff08c77760945729ee8517d6de31ce5', 'request_id': 'req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:42.353961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff08c77760945729ee8517d6de31ce5', 'request_id': 'req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:42.365883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] The token cache is being invalidate because project bff08c77760945729ee8517d6de31ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:42.366173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:42.366831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:42.368393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1094/2207] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:42 2022] DELETE /identity/v3/projects/bff08c77760945729ee8517d6de31ce5 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:42.374951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a075b6c-1946-41cb-be05-f16521fee0f6 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:42.386531 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a075b6c-1946-41cb-be05-f16521fee0f6 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:42.442775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447db0ff-ea95-4bb3-a16d-075637ce611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:42.443327 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db0ff-ea95-4bb3-a16d-075637ce611f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:42.443605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db0ff-ea95-4bb3-a16d-075637ce611f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:42.443972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db0ff-ea95-4bb3-a16d-075637ce611f admin admin] PATH_INFO: `/v3/users/0b5fbd4828104cf99918ec9dcf643297` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:42.445159 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447db0ff-ea95-4bb3-a16d-075637ce611f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5fbd4828104cf99918ec9dcf643297)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:42.447111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447db0ff-ea95-4bb3-a16d-075637ce611f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:42.618345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-447db0ff-ea95-4bb3-a16d-075637ce611f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5fbd4828104cf99918ec9dcf643297'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:42.722798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-447db0ff-ea95-4bb3-a16d-075637ce611f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5fbd4828104cf99918ec9dcf643297'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:42.723502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-447db0ff-ea95-4bb3-a16d-075637ce611f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5fbd4828104cf99918ec9dcf643297'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:42.724138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-447db0ff-ea95-4bb3-a16d-075637ce611f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5fbd4828104cf99918ec9dcf643297'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:42.725642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:42.739518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-447db0ff-ea95-4bb3-a16d-075637ce611f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5fbd4828104cf99918ec9dcf643297'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:42.757849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1114/2208] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:42 2022] DELETE /identity/v3/users/0b5fbd4828104cf99918ec9dcf643297 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:43.243242 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:43.244672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:43.271248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81112e71-8dfc-4c45-8f4e-c8deaafaee1c admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:43.277264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:43.406999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:43.407932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:43.408317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:43.409580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] PATH_INFO: `/v3/projects/42628cea306d46588e875127f6f958b8` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:43.414564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42628cea306d46588e875127f6f958b8)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:43.424568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:43.566441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42628cea306d46588e875127f6f958b8', 'request_id': 'req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:43.637331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42628cea306d46588e875127f6f958b8', 'request_id': 'req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:43.644900 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42628cea306d46588e875127f6f958b8', 'request_id': 'req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:43.655473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] The token cache is being invalidate because project 42628cea306d46588e875127f6f958b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:43.655794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:43.656412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-3de7a0bb-bca9-4591-9ac0-ad626e3aa5e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:43.657911 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1095/2209] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:43 2022] DELETE /identity/v3/projects/42628cea306d46588e875127f6f958b8 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:43.664592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33fa7-ac14-4e5f-adad-a652debf19b2 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:43.664592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33fa7-ac14-4e5f-adad-a652debf19b2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:43.664592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff33fa7-ac14-4e5f-adad-a652debf19b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:43.684532 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-3ff33fa7-ac14-4e5f-adad-a652debf19b2 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:43.685734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-3ff33fa7-ac14-4e5f-adad-a652debf19b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:43.756654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1115/2210] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:43.764773 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d40e-282d-465b-b6e2-5515e2be08bb None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:43.764997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d40e-282d-465b-b6e2-5515e2be08bb None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:43.765338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2d40e-282d-465b-b6e2-5515e2be08bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:43.785169 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-71d2d40e-282d-465b-b6e2-5515e2be08bb None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:43.786377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-71d2d40e-282d-465b-b6e2-5515e2be08bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:43.794892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1096/2211] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:43.799163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff33fa7-ac14-4e5f-adad-a652debf19b2 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:43.810436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff33fa7-ac14-4e5f-adad-a652debf19b2 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:43.815146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826d130-c2e8-41aa-a990-5b7aa80b02d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:43.816448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826d130-c2e8-41aa-a990-5b7aa80b02d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:43.816745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826d130-c2e8-41aa-a990-5b7aa80b02d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:43.816996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826d130-c2e8-41aa-a990-5b7aa80b02d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:43.817676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826d130-c2e8-41aa-a990-5b7aa80b02d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:43.817973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826d130-c2e8-41aa-a990-5b7aa80b02d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:43.819564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826d130-c2e8-41aa-a990-5b7aa80b02d2 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:43.831242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1116/2212] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:24:43.835608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2d40e-282d-465b-b6e2-5515e2be08bb None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:43.845406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2d40e-282d-465b-b6e2-5515e2be08bb None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:43.851352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e819aa-c8ce-41a0-8978-3317b3bad2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:43.852032 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e819aa-c8ce-41a0-8978-3317b3bad2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:43.852312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e819aa-c8ce-41a0-8978-3317b3bad2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:43.852583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e819aa-c8ce-41a0-8978-3317b3bad2c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:43.853259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e819aa-c8ce-41a0-8978-3317b3bad2c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:43.856004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e819aa-c8ce-41a0-8978-3317b3bad2c3 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:43.967487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1097/2213] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:43 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:43.971898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826d130-c2e8-41aa-a990-5b7aa80b02d2 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:43.984132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826d130-c2e8-41aa-a990-5b7aa80b02d2 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:44.070404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50bfd-4b5c-43ce-80d9-587b66dcbfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:44.070404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50bfd-4b5c-43ce-80d9-587b66dcbfbd admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:44.070677 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50bfd-4b5c-43ce-80d9-587b66dcbfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:44.071018 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa50bfd-4b5c-43ce-80d9-587b66dcbfbd admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:44.071681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa50bfd-4b5c-43ce-80d9-587b66dcbfbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:44.073712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa50bfd-4b5c-43ce-80d9-587b66dcbfbd admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:44.162056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1117/2214] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:43 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:44.166007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e819aa-c8ce-41a0-8978-3317b3bad2c3 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:44.178776 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e819aa-c8ce-41a0-8978-3317b3bad2c3 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:44.184868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ec4e0-b549-432c-98e3-99bd96b390d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:44.185672 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ec4e0-b549-432c-98e3-99bd96b390d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:44.186013 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ec4e0-b549-432c-98e3-99bd96b390d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:44.186359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ec4e0-b549-432c-98e3-99bd96b390d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:44.187081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ec4e0-b549-432c-98e3-99bd96b390d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:44.188967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ec4e0-b549-432c-98e3-99bd96b390d8 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:44.200521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1098/2215] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:44.204773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50bfd-4b5c-43ce-80d9-587b66dcbfbd admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:44.220147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa50bfd-4b5c-43ce-80d9-587b66dcbfbd admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:44.225478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd1aad-3b71-426b-9810-c0ce704da68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:44.226086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd1aad-3b71-426b-9810-c0ce704da68b admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:44.226372 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd1aad-3b71-426b-9810-c0ce704da68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:44.226652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd1aad-3b71-426b-9810-c0ce704da68b admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:44.227299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd1aad-3b71-426b-9810-c0ce704da68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:44.229054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bd1aad-3b71-426b-9810-c0ce704da68b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:44.249553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1118/2216] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:44.254052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ec4e0-b549-432c-98e3-99bd96b390d8 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:44.265621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ec4e0-b549-432c-98e3-99bd96b390d8 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:44.272760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c446db-ad9e-4fe3-a3f8-1141387ba50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:44.273453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c446db-ad9e-4fe3-a3f8-1141387ba50b admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:44.273767 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c446db-ad9e-4fe3-a3f8-1141387ba50b admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:44.274048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c446db-ad9e-4fe3-a3f8-1141387ba50b admin admin] PATH_INFO: `/v3/projects/cda570169e0447bfa8dadba146d8e358/users/426bb904b5524c84ab33ad488518d41c/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:44.295012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c446db-ad9e-4fe3-a3f8-1141387ba50b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda570169e0447bfa8dadba146d8e358, user_id=426bb904b5524c84ab33ad488518d41c, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:44.297860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c446db-ad9e-4fe3-a3f8-1141387ba50b admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:44.388714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1099/2217] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:44 2022] PUT /identity/v3/projects/cda570169e0447bfa8dadba146d8e358/users/426bb904b5524c84ab33ad488518d41c/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:44.401462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba942474-bf96-4913-a098-b59c99bd2478 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:44.401834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba942474-bf96-4913-a098-b59c99bd2478 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:44.402176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba942474-bf96-4913-a098-b59c99bd2478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:44.431241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ba942474-bf96-4913-a098-b59c99bd2478 None None] MFA Rules not processed for user `426bb904b5524c84ab33ad488518d41c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:44.432502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ba942474-bf96-4913-a098-b59c99bd2478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:44.464725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:44.507984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1119/2218] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:44.522743 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c446db-ad9e-4fe3-a3f8-1141387ba50b admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:44.524910 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-40c446db-ad9e-4fe3-a3f8-1141387ba50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:44.576548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:44.600301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c446db-ad9e-4fe3-a3f8-1141387ba50b admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:44.604270 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436324e2-038f-45fb-b6f7-4d0e4d47e914 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:44.605003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436324e2-038f-45fb-b6f7-4d0e4d47e914 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:44.605511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436324e2-038f-45fb-b6f7-4d0e4d47e914 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:44.606089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436324e2-038f-45fb-b6f7-4d0e4d47e914 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:44.617278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436324e2-038f-45fb-b6f7-4d0e4d47e914 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:44.618719 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436324e2-038f-45fb-b6f7-4d0e4d47e914 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:44.628398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1100/2219] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:24:44.671552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba942474-bf96-4913-a098-b59c99bd2478 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:44.758130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba942474-bf96-4913-a098-b59c99bd2478 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:44.777582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:44.778245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:44.778573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:44.778826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] PATH_INFO: `/v3/users/426bb904b5524c84ab33ad488518d41c` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:44.780079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=426bb904b5524c84ab33ad488518d41c)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:44.782517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:45.038563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '426bb904b5524c84ab33ad488518d41c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:45.130956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '426bb904b5524c84ab33ad488518d41c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:45.131696 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '426bb904b5524c84ab33ad488518d41c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:45.132535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '426bb904b5524c84ab33ad488518d41c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:45.140225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '426bb904b5524c84ab33ad488518d41c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:45.166420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1120/2220] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:44 2022] DELETE /identity/v3/users/426bb904b5524c84ab33ad488518d41c => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:45.171370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26099a88-5ac5-4a5a-aaaf-f84200e4d2ee None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:45.171656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26099a88-5ac5-4a5a-aaaf-f84200e4d2ee None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:45.171967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26099a88-5ac5-4a5a-aaaf-f84200e4d2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:45.191631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-26099a88-5ac5-4a5a-aaaf-f84200e4d2ee None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:45.192930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-26099a88-5ac5-4a5a-aaaf-f84200e4d2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:45.259954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1101/2221] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:45.271027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:45.272445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:45.329425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:45.339999 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fc81d-c21d-43e2-bd57-0739ebac2a5a admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:45.343804 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdb9da-b7c0-423b-81ee-49b057dc49a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:45.344450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdb9da-b7c0-423b-81ee-49b057dc49a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:45.344719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdb9da-b7c0-423b-81ee-49b057dc49a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:45.344989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdb9da-b7c0-423b-81ee-49b057dc49a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:45.354875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdb9da-b7c0-423b-81ee-49b057dc49a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:45.356336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdb9da-b7c0-423b-81ee-49b057dc49a1 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:45.365540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1121/2222] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:45.650181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26099a88-5ac5-4a5a-aaaf-f84200e4d2ee None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:45.652125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-26099a88-5ac5-4a5a-aaaf-f84200e4d2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:45.666339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26099a88-5ac5-4a5a-aaaf-f84200e4d2ee None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:45.674676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a78e1b-f05f-447b-8732-6ef91e486153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:45.677645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a78e1b-f05f-447b-8732-6ef91e486153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:45.677645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a78e1b-f05f-447b-8732-6ef91e486153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:45.677645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a78e1b-f05f-447b-8732-6ef91e486153 admin admin] PATH_INFO: `/v3/users/779987ce97764b18a7207c2523365c23` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:45.679069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a78e1b-f05f-447b-8732-6ef91e486153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=779987ce97764b18a7207c2523365c23)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:45.681188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a78e1b-f05f-447b-8732-6ef91e486153 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:45.917823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdb9da-b7c0-423b-81ee-49b057dc49a1 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:45.919384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-37bdb9da-b7c0-423b-81ee-49b057dc49a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:45.950837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdb9da-b7c0-423b-81ee-49b057dc49a1 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:45.960109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:45.961097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:45.961361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:45.961726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] PATH_INFO: `/v3/projects/cda570169e0447bfa8dadba146d8e358` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:45.963022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda570169e0447bfa8dadba146d8e358)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:45.966635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-33a78e1b-f05f-447b-8732-6ef91e486153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '779987ce97764b18a7207c2523365c23'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:45.966777 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:46.042642 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-33a78e1b-f05f-447b-8732-6ef91e486153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '779987ce97764b18a7207c2523365c23'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.043287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-33a78e1b-f05f-447b-8732-6ef91e486153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '779987ce97764b18a7207c2523365c23'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.043926 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-33a78e1b-f05f-447b-8732-6ef91e486153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '779987ce97764b18a7207c2523365c23'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.052181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-33a78e1b-f05f-447b-8732-6ef91e486153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '779987ce97764b18a7207c2523365c23'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.071059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1102/2223] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:45 2022] DELETE /identity/v3/users/779987ce97764b18a7207c2523365c23 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:46.075896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789b076-0b51-4d5f-9925-35561debc934 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:46.076318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789b076-0b51-4d5f-9925-35561debc934 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:46.076714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789b076-0b51-4d5f-9925-35561debc934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:46.092577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda570169e0447bfa8dadba146d8e358', 'request_id': 'req-940f2959-e164-4cf3-9758-14f68242ee22'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.096113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-6789b076-0b51-4d5f-9925-35561debc934 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:46.097373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6789b076-0b51-4d5f-9925-35561debc934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:46.098307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda570169e0447bfa8dadba146d8e358', 'request_id': 'req-940f2959-e164-4cf3-9758-14f68242ee22'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.141283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:46.161195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1103/2224] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:46.166102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789b076-0b51-4d5f-9925-35561debc934 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:46.167404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6789b076-0b51-4d5f-9925-35561debc934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:46.208330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda570169e0447bfa8dadba146d8e358', 'request_id': 'req-940f2959-e164-4cf3-9758-14f68242ee22'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.220837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] The token cache is being invalidate because project cda570169e0447bfa8dadba146d8e358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:46.221274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.221987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.223813 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1122/2225] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:45 2022] DELETE /identity/v3/projects/cda570169e0447bfa8dadba146d8e358 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:46.224655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789b076-0b51-4d5f-9925-35561debc934 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:46.228515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a74a5-69e2-4f58-8229-b27f4c68282b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:46.229154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a74a5-69e2-4f58-8229-b27f4c68282b service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:46.229439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a74a5-69e2-4f58-8229-b27f4c68282b service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:46.229821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a74a5-69e2-4f58-8229-b27f4c68282b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:46.231715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ff6a74a5-69e2-4f58-8229-b27f4c68282b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:46.246243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a74a5-69e2-4f58-8229-b27f4c68282b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:46.247822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a74a5-69e2-4f58-8229-b27f4c68282b service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:46.259471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1104/2226] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:24:46.686866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:46.688416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:46.709051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f2959-e164-4cf3-9758-14f68242ee22 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:46.716852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:46.717976 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:46.718365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:46.718743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] PATH_INFO: `/v3/projects/5d62a93c2cfb4c3681e08d4e9457f113` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:46.727710 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d62a93c2cfb4c3681e08d4e9457f113)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:46.731065 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:46.871013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d62a93c2cfb4c3681e08d4e9457f113', 'request_id': 'req-07a7762a-68c9-4709-83dd-36760d20b37f'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.876541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d62a93c2cfb4c3681e08d4e9457f113', 'request_id': 'req-07a7762a-68c9-4709-83dd-36760d20b37f'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.974144 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d62a93c2cfb4c3681e08d4e9457f113', 'request_id': 'req-07a7762a-68c9-4709-83dd-36760d20b37f'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.986618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] The token cache is being invalidate because project 5d62a93c2cfb4c3681e08d4e9457f113 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:46.986937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.987714 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:46.989395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1123/2227] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:46 2022] DELETE /identity/v3/projects/5d62a93c2cfb4c3681e08d4e9457f113 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:46.999567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a74a5-69e2-4f58-8229-b27f4c68282b service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:47.009834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a74a5-69e2-4f58-8229-b27f4c68282b service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:47.076676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4d958-6cf7-44df-81f2-3e5a9b6fb969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:47.077408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4d958-6cf7-44df-81f2-3e5a9b6fb969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:47.077695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4d958-6cf7-44df-81f2-3e5a9b6fb969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:47.077977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4d958-6cf7-44df-81f2-3e5a9b6fb969 admin admin] PATH_INFO: `/v3/users/dc4eed6e32e1401db4f0f381f6eb602d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:47.079123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4d958-6cf7-44df-81f2-3e5a9b6fb969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc4eed6e32e1401db4f0f381f6eb602d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:47.081685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4d958-6cf7-44df-81f2-3e5a9b6fb969 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:47.323857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c5f4d958-6cf7-44df-81f2-3e5a9b6fb969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc4eed6e32e1401db4f0f381f6eb602d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:47.390921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c5f4d958-6cf7-44df-81f2-3e5a9b6fb969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc4eed6e32e1401db4f0f381f6eb602d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:47.392024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c5f4d958-6cf7-44df-81f2-3e5a9b6fb969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc4eed6e32e1401db4f0f381f6eb602d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:47.392791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c5f4d958-6cf7-44df-81f2-3e5a9b6fb969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc4eed6e32e1401db4f0f381f6eb602d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:47.401733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c5f4d958-6cf7-44df-81f2-3e5a9b6fb969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc4eed6e32e1401db4f0f381f6eb602d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:47.420677 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1105/2228] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:46 2022] DELETE /identity/v3/users/dc4eed6e32e1401db4f0f381f6eb602d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:47.844067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:47.845483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:47.860618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7762a-68c9-4709-83dd-36760d20b37f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:47.935179 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:47.942412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:47.943049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:47.943328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:47.943594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] PATH_INFO: `/v3/projects/8cd6cebf3cc94c38ad35bbb1bd1d9469` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:47.944734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cd6cebf3cc94c38ad35bbb1bd1d9469)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:47.946619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:48.053273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cd6cebf3cc94c38ad35bbb1bd1d9469', 'request_id': 'req-f451fcb6-fa12-46ea-82c1-5425234cf242'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:48.059073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cd6cebf3cc94c38ad35bbb1bd1d9469', 'request_id': 'req-f451fcb6-fa12-46ea-82c1-5425234cf242'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:48.126109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cd6cebf3cc94c38ad35bbb1bd1d9469', 'request_id': 'req-f451fcb6-fa12-46ea-82c1-5425234cf242'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:48.138696 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] The token cache is being invalidate because project 8cd6cebf3cc94c38ad35bbb1bd1d9469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:48.139215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:48.139977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f451fcb6-fa12-46ea-82c1-5425234cf242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:48.142235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1124/2229] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:47 2022] DELETE /identity/v3/projects/8cd6cebf3cc94c38ad35bbb1bd1d9469 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:48.148088 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eedf44-f055-45cb-acb8-749b90e18457 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:48.148593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eedf44-f055-45cb-acb8-749b90e18457 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:48.148729 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eedf44-f055-45cb-acb8-749b90e18457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:48.173567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-b7eedf44-f055-45cb-acb8-749b90e18457 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:48.175535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b7eedf44-f055-45cb-acb8-749b90e18457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:48.181968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:48.244216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1106/2230] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:48.253260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b5501-15d0-4f4c-a31d-9316031330c8 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:48.253539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b5501-15d0-4f4c-a31d-9316031330c8 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:48.253816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b5501-15d0-4f4c-a31d-9316031330c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:48.274915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e17b5501-15d0-4f4c-a31d-9316031330c8 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:48.276156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e17b5501-15d0-4f4c-a31d-9316031330c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:48.284681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1125/2231] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:24:48.289061 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eedf44-f055-45cb-acb8-749b90e18457 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:48.298773 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eedf44-f055-45cb-acb8-749b90e18457 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:48.303385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafe08a-d331-415a-b472-578a7207662d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:48.304045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe08a-d331-415a-b472-578a7207662d admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:48.304324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe08a-d331-415a-b472-578a7207662d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:48.304594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafe08a-d331-415a-b472-578a7207662d admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:48.305235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafe08a-d331-415a-b472-578a7207662d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:48.305535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafe08a-d331-415a-b472-578a7207662d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:48.307165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafe08a-d331-415a-b472-578a7207662d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:48.318500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1107/2232] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:24:48.322635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b5501-15d0-4f4c-a31d-9316031330c8 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:48.332134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b5501-15d0-4f4c-a31d-9316031330c8 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:48.336681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b03ea-6e94-4545-96f4-aa6c83097d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:48.337267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b03ea-6e94-4545-96f4-aa6c83097d5f admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:48.337540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b03ea-6e94-4545-96f4-aa6c83097d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:48.337858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b03ea-6e94-4545-96f4-aa6c83097d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:48.338430 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b03ea-6e94-4545-96f4-aa6c83097d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:48.340343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b03ea-6e94-4545-96f4-aa6c83097d5f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:48.441828 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1126/2233] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:48 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:48.442746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafe08a-d331-415a-b472-578a7207662d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:48.453703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafe08a-d331-415a-b472-578a7207662d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:48.523138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18559b04-eb3c-40a9-bafa-8c3ad7c55401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:48.523739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18559b04-eb3c-40a9-bafa-8c3ad7c55401 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:48.524049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18559b04-eb3c-40a9-bafa-8c3ad7c55401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:48.524325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18559b04-eb3c-40a9-bafa-8c3ad7c55401 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:48.524972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18559b04-eb3c-40a9-bafa-8c3ad7c55401 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:48.526886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18559b04-eb3c-40a9-bafa-8c3ad7c55401 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:48.599452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1108/2234] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:48 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:48.603725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b03ea-6e94-4545-96f4-aa6c83097d5f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:48.621517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b03ea-6e94-4545-96f4-aa6c83097d5f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:48.627136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29d841-b06e-4d76-aae0-d381d98f420c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:48.628369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29d841-b06e-4d76-aae0-d381d98f420c admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:48.628726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29d841-b06e-4d76-aae0-d381d98f420c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:48.629127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29d841-b06e-4d76-aae0-d381d98f420c admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:48.629848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29d841-b06e-4d76-aae0-d381d98f420c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:48.631570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29d841-b06e-4d76-aae0-d381d98f420c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:48.653570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1127/2235] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:48.657989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18559b04-eb3c-40a9-bafa-8c3ad7c55401 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:48.668373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18559b04-eb3c-40a9-bafa-8c3ad7c55401 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:48.673686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c5d48-93ad-49f5-994b-415930ed8f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:48.674290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5d48-93ad-49f5-994b-415930ed8f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:48.674570 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5d48-93ad-49f5-994b-415930ed8f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:48.674887 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5d48-93ad-49f5-994b-415930ed8f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:48.676749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c5d48-93ad-49f5-994b-415930ed8f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:48.677291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c5d48-93ad-49f5-994b-415930ed8f2e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:48.692698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1109/2236] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:48.693604 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29d841-b06e-4d76-aae0-d381d98f420c admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:48.705097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29d841-b06e-4d76-aae0-d381d98f420c admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:48.710446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d0aca-46ea-4cc7-96d3-78cf3f197020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:48.711100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d0aca-46ea-4cc7-96d3-78cf3f197020 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:48.711380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d0aca-46ea-4cc7-96d3-78cf3f197020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:48.711654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d0aca-46ea-4cc7-96d3-78cf3f197020 admin admin] PATH_INFO: `/v3/projects/1d42791837334f048f85743971d36328/users/5d43c911a4aa4096aa5ed550625f6100/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:48.726671 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d0aca-46ea-4cc7-96d3-78cf3f197020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d42791837334f048f85743971d36328, user_id=5d43c911a4aa4096aa5ed550625f6100, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:48.729478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d0aca-46ea-4cc7-96d3-78cf3f197020 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:48.801434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1128/2237] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:48 2022] PUT /identity/v3/projects/1d42791837334f048f85743971d36328/users/5d43c911a4aa4096aa5ed550625f6100/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:48.811322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c170157-66a3-4712-8a71-ff41d46d8926 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:48.811708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c170157-66a3-4712-8a71-ff41d46d8926 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:48.812016 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c170157-66a3-4712-8a71-ff41d46d8926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:48.843148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-2c170157-66a3-4712-8a71-ff41d46d8926 None None] MFA Rules not processed for user `5d43c911a4aa4096aa5ed550625f6100`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:48.844402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2c170157-66a3-4712-8a71-ff41d46d8926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:48.924422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1110/2238] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:48.929798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d0aca-46ea-4cc7-96d3-78cf3f197020 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:49.000188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d0aca-46ea-4cc7-96d3-78cf3f197020 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:49.006227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bfbc1-c35a-4b11-adc2-0064abdcf829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:49.006897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bfbc1-c35a-4b11-adc2-0064abdcf829 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:49.007171 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bfbc1-c35a-4b11-adc2-0064abdcf829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:49.007444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2bfbc1-c35a-4b11-adc2-0064abdcf829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:49.008110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bfbc1-c35a-4b11-adc2-0064abdcf829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:49.010016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2bfbc1-c35a-4b11-adc2-0064abdcf829 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:49.121399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1129/2239] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:48 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:49.125362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c170157-66a3-4712-8a71-ff41d46d8926 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:49.143567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c170157-66a3-4712-8a71-ff41d46d8926 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:49.149856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:49.231452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4916e7-5e22-4080-9510-5fca1ae20b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:49.236746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4916e7-5e22-4080-9510-5fca1ae20b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:49.236746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4916e7-5e22-4080-9510-5fca1ae20b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:49.236746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4916e7-5e22-4080-9510-5fca1ae20b31 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:49.236746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4916e7-5e22-4080-9510-5fca1ae20b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:49.236746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4916e7-5e22-4080-9510-5fca1ae20b31 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:49.322643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1111/2240] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:49 2022] POST /identity/v3/users => generated 298 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:49.323915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bfbc1-c35a-4b11-adc2-0064abdcf829 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:49.336203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2bfbc1-c35a-4b11-adc2-0064abdcf829 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:49.342256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6afe1f-0a05-452c-9233-30a7c9782fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:49.342989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6afe1f-0a05-452c-9233-30a7c9782fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:49.343377 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6afe1f-0a05-452c-9233-30a7c9782fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:49.343781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6afe1f-0a05-452c-9233-30a7c9782fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:49.344544 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6afe1f-0a05-452c-9233-30a7c9782fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:49.346331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6afe1f-0a05-452c-9233-30a7c9782fd4 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:49.348669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:49.366215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1130/2241] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:49.366914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4916e7-5e22-4080-9510-5fca1ae20b31 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:49.381719 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4916e7-5e22-4080-9510-5fca1ae20b31 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:49.387626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199154eb-6942-4baa-8cf0-eed9a5de61c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:49.388333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199154eb-6942-4baa-8cf0-eed9a5de61c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:49.388611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199154eb-6942-4baa-8cf0-eed9a5de61c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:49.388886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199154eb-6942-4baa-8cf0-eed9a5de61c5 admin admin] PATH_INFO: `/v3/projects/44c34c23a39e4c8d99c6bf6c6237da7c/users/8989d044bcd74c89ae939b839bece8d7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:49.404593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199154eb-6942-4baa-8cf0-eed9a5de61c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c34c23a39e4c8d99c6bf6c6237da7c, user_id=8989d044bcd74c89ae939b839bece8d7, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:49.407307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199154eb-6942-4baa-8cf0-eed9a5de61c5 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:49.505532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1112/2242] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:49 2022] PUT /identity/v3/projects/44c34c23a39e4c8d99c6bf6c6237da7c/users/8989d044bcd74c89ae939b839bece8d7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:49.509459 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6afe1f-0a05-452c-9233-30a7c9782fd4 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:49.520695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6afe1f-0a05-452c-9233-30a7c9782fd4 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:49.598079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88ecc5-7632-4a67-a75f-8b6d75f8d06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:49.598993 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88ecc5-7632-4a67-a75f-8b6d75f8d06c admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:49.599293 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88ecc5-7632-4a67-a75f-8b6d75f8d06c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:49.599598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88ecc5-7632-4a67-a75f-8b6d75f8d06c admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:49.600273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88ecc5-7632-4a67-a75f-8b6d75f8d06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:49.601893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88ecc5-7632-4a67-a75f-8b6d75f8d06c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:49.611934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1131/2243] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:24:49.616598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199154eb-6942-4baa-8cf0-eed9a5de61c5 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:49.626929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199154eb-6942-4baa-8cf0-eed9a5de61c5 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:49.631618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e0d62-3409-4964-94c1-5bbd6f57bfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:49.632342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e0d62-3409-4964-94c1-5bbd6f57bfd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:49.632589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e0d62-3409-4964-94c1-5bbd6f57bfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:49.632865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e0d62-3409-4964-94c1-5bbd6f57bfd6 admin admin] PATH_INFO: `/v3/projects/44c34c23a39e4c8d99c6bf6c6237da7c/users/8989d044bcd74c89ae939b839bece8d7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:49.634784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e0d62-3409-4964-94c1-5bbd6f57bfd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44c34c23a39e4c8d99c6bf6c6237da7c, user_id=8989d044bcd74c89ae939b839bece8d7, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:49.637456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e0d62-3409-4964-94c1-5bbd6f57bfd6 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:49.649514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1113/2244] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:49 2022] PUT /identity/v3/projects/44c34c23a39e4c8d99c6bf6c6237da7c/users/8989d044bcd74c89ae939b839bece8d7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:49.663132 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bd723-d3f1-47cf-aff0-4965eafbe356 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:49.663726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bd723-d3f1-47cf-aff0-4965eafbe356 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:49.664189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bd723-d3f1-47cf-aff0-4965eafbe356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:49.696114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-410bd723-d3f1-47cf-aff0-4965eafbe356 None None] MFA Rules not processed for user `8989d044bcd74c89ae939b839bece8d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:49.698039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-410bd723-d3f1-47cf-aff0-4965eafbe356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:49.804099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1132/2245] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:49.814732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e0d62-3409-4964-94c1-5bbd6f57bfd6 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:49.816809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e91e0d62-3409-4964-94c1-5bbd6f57bfd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:49.884735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e0d62-3409-4964-94c1-5bbd6f57bfd6 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:49.888604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50e9b5-04c2-4ee3-9ff4-4890514fdbde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:49.889803 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50e9b5-04c2-4ee3-9ff4-4890514fdbde service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:49.890085 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50e9b5-04c2-4ee3-9ff4-4890514fdbde service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:49.890361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50e9b5-04c2-4ee3-9ff4-4890514fdbde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:49.900575 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50e9b5-04c2-4ee3-9ff4-4890514fdbde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:49.902024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50e9b5-04c2-4ee3-9ff4-4890514fdbde service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:49.912495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1114/2246] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:50.039299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bd723-d3f1-47cf-aff0-4965eafbe356 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:50.049741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410bd723-d3f1-47cf-aff0-4965eafbe356 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:50.053491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28462aee-f200-4f0a-b69c-1f7bcfdfea9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:50.054155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28462aee-f200-4f0a-b69c-1f7bcfdfea9e service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:50.054442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28462aee-f200-4f0a-b69c-1f7bcfdfea9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:50.054718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28462aee-f200-4f0a-b69c-1f7bcfdfea9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:50.112419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28462aee-f200-4f0a-b69c-1f7bcfdfea9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:50.113925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28462aee-f200-4f0a-b69c-1f7bcfdfea9e service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:50.123120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1133/2247] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:55.461391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50e9b5-04c2-4ee3-9ff4-4890514fdbde service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:55.523436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50e9b5-04c2-4ee3-9ff4-4890514fdbde service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:55.527991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:55.528611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:55.528889 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:55.529162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] PATH_INFO: `/v3/users/5d43c911a4aa4096aa5ed550625f6100` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:55.530269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d43c911a4aa4096aa5ed550625f6100)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:55.532209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:55.534994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:55.744310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d43c911a4aa4096aa5ed550625f6100'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:55.810603 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d43c911a4aa4096aa5ed550625f6100'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:55.811241 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d43c911a4aa4096aa5ed550625f6100'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:55.811873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d43c911a4aa4096aa5ed550625f6100'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:55.819279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d43c911a4aa4096aa5ed550625f6100'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:55.838640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1115/2248] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:55 2022] DELETE /identity/v3/users/5d43c911a4aa4096aa5ed550625f6100 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:55.843072 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabce7f-ca57-4594-b6b9-a7038af66179 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:55.843523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabce7f-ca57-4594-b6b9-a7038af66179 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:55.843685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabce7f-ca57-4594-b6b9-a7038af66179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:55.864488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-4dabce7f-ca57-4594-b6b9-a7038af66179 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:55.865680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4dabce7f-ca57-4594-b6b9-a7038af66179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:55.874951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:55.926325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1134/2249] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:55.936198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:55.937627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:55.995301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9dbea-89ac-4c01-892e-17844b932253 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:55.999151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbcfbd-92f6-4600-bebc-a55d3db1945f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:55.999784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbcfbd-92f6-4600-bebc-a55d3db1945f service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:56.000070 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbcfbd-92f6-4600-bebc-a55d3db1945f service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:56.000360 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecbcfbd-92f6-4600-bebc-a55d3db1945f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:56.013448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbcfbd-92f6-4600-bebc-a55d3db1945f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:56.014918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecbcfbd-92f6-4600-bebc-a55d3db1945f service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:56.025466 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1116/2250] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:24:56.779514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabce7f-ca57-4594-b6b9-a7038af66179 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:56.784252 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4dabce7f-ca57-4594-b6b9-a7038af66179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:56.796573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dabce7f-ca57-4594-b6b9-a7038af66179 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:56.801619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:56.802290 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:56.803082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:56.803527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] PATH_INFO: `/v3/projects/1d42791837334f048f85743971d36328` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:56.804792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d42791837334f048f85743971d36328)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:56.806747 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:56.922891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d42791837334f048f85743971d36328', 'request_id': 'req-38bd2bf3-c593-4762-9ead-805f589e9653'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:56.928940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d42791837334f048f85743971d36328', 'request_id': 'req-38bd2bf3-c593-4762-9ead-805f589e9653'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:56.999232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d42791837334f048f85743971d36328', 'request_id': 'req-38bd2bf3-c593-4762-9ead-805f589e9653'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:57.013189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] The token cache is being invalidate because project 1d42791837334f048f85743971d36328 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:57.013476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:57.014109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:57.015574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1135/2251] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:56 2022] DELETE /identity/v3/projects/1d42791837334f048f85743971d36328 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:57.019507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbcfbd-92f6-4600-bebc-a55d3db1945f service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:57.029066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecbcfbd-92f6-4600-bebc-a55d3db1945f service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:57.094172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013b250-bfa9-4ae5-92c9-544e615bdf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:57.096951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b250-bfa9-4ae5-92c9-544e615bdf0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:57.096951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b250-bfa9-4ae5-92c9-544e615bdf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:57.096951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b250-bfa9-4ae5-92c9-544e615bdf0e admin admin] PATH_INFO: `/v3/users/8989d044bcd74c89ae939b839bece8d7` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:57.097907 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013b250-bfa9-4ae5-92c9-544e615bdf0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8989d044bcd74c89ae939b839bece8d7)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:57.099924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013b250-bfa9-4ae5-92c9-544e615bdf0e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:57.196749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:57.306567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1013b250-bfa9-4ae5-92c9-544e615bdf0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8989d044bcd74c89ae939b839bece8d7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:57.370531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1013b250-bfa9-4ae5-92c9-544e615bdf0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8989d044bcd74c89ae939b839bece8d7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:57.371198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1013b250-bfa9-4ae5-92c9-544e615bdf0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8989d044bcd74c89ae939b839bece8d7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:57.371898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1013b250-bfa9-4ae5-92c9-544e615bdf0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8989d044bcd74c89ae939b839bece8d7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:57.379075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1013b250-bfa9-4ae5-92c9-544e615bdf0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8989d044bcd74c89ae939b839bece8d7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:57.396953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1117/2252] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:24:57 2022] DELETE /identity/v3/users/8989d044bcd74c89ae939b839bece8d7 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:57.874078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:57.875533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:57.900524 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd2bf3-c593-4762-9ead-805f589e9653 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:57.959521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:57.960206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:57.960498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:57.960891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] PATH_INFO: `/v3/projects/44c34c23a39e4c8d99c6bf6c6237da7c` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:57.962017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=44c34c23a39e4c8d99c6bf6c6237da7c)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:57.963988 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:58.062685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:58.069637 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44c34c23a39e4c8d99c6bf6c6237da7c', 'request_id': 'req-f443cb66-4681-49ac-930d-74b47c067aee'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:58.075156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44c34c23a39e4c8d99c6bf6c6237da7c', 'request_id': 'req-f443cb66-4681-49ac-930d-74b47c067aee'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:58.135959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44c34c23a39e4c8d99c6bf6c6237da7c', 'request_id': 'req-f443cb66-4681-49ac-930d-74b47c067aee'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:58.148351 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] The token cache is being invalidate because project 44c34c23a39e4c8d99c6bf6c6237da7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:24:58.148670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:58.149313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f443cb66-4681-49ac-930d-74b47c067aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:24:58.150862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1136/2253] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:24:57 2022] DELETE /identity/v3/projects/44c34c23a39e4c8d99c6bf6c6237da7c => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:58.155778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70829a-0ce8-49a6-995a-49149d8db0e1 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:58.156238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70829a-0ce8-49a6-995a-49149d8db0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:58.156342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70829a-0ce8-49a6-995a-49149d8db0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:58.176722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-dc70829a-0ce8-49a6-995a-49149d8db0e1 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:58.177948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-dc70829a-0ce8-49a6-995a-49149d8db0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:58.235319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1118/2254] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:58.243575 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d5ec1-47c0-4e4c-9504-65479a4af60b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:58.243891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d5ec1-47c0-4e4c-9504-65479a4af60b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:58.244285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d5ec1-47c0-4e4c-9504-65479a4af60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:58.264147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-324d5ec1-47c0-4e4c-9504-65479a4af60b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:58.265267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-324d5ec1-47c0-4e4c-9504-65479a4af60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:58.273058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1137/2255] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:58.277191 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70829a-0ce8-49a6-995a-49149d8db0e1 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:58.287305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70829a-0ce8-49a6-995a-49149d8db0e1 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:58.291917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dd54c-d281-4eff-b573-827ab78cf98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:58.292510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dd54c-d281-4eff-b573-827ab78cf98c admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:58.292787 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dd54c-d281-4eff-b573-827ab78cf98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:58.293063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dd54c-d281-4eff-b573-827ab78cf98c admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:58.293699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dd54c-d281-4eff-b573-827ab78cf98c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:24:58.294003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dd54c-d281-4eff-b573-827ab78cf98c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:58.295875 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035dd54c-d281-4eff-b573-827ab78cf98c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:58.307850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1119/2256] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:24:58.311958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d5ec1-47c0-4e4c-9504-65479a4af60b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:58.321326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d5ec1-47c0-4e4c-9504-65479a4af60b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:58.326057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c54d6-eb93-4880-ac0b-f16e8cb42cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:58.326647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c54d6-eb93-4880-ac0b-f16e8cb42cae admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:58.326920 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c54d6-eb93-4880-ac0b-f16e8cb42cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:58.327189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c54d6-eb93-4880-ac0b-f16e8cb42cae admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:58.328052 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c54d6-eb93-4880-ac0b-f16e8cb42cae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:58.330133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c54d6-eb93-4880-ac0b-f16e8cb42cae admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:58.432552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1138/2257] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:58 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:58.436839 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dd54c-d281-4eff-b573-827ab78cf98c admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:58.447110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dd54c-d281-4eff-b573-827ab78cf98c admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:58.502020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85970f18-e9e6-4798-93c8-fa98b6640a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:58.502647 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85970f18-e9e6-4798-93c8-fa98b6640a71 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:58.502927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85970f18-e9e6-4798-93c8-fa98b6640a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:58.503193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85970f18-e9e6-4798-93c8-fa98b6640a71 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:58.503851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85970f18-e9e6-4798-93c8-fa98b6640a71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:58.505745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85970f18-e9e6-4798-93c8-fa98b6640a71 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:58.587135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1120/2258] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:58 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:58.591095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c54d6-eb93-4880-ac0b-f16e8cb42cae admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:58.602078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c54d6-eb93-4880-ac0b-f16e8cb42cae admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:58.606731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dd363-8286-45b2-8b1a-2800923ad8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:58.607362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dd363-8286-45b2-8b1a-2800923ad8fc admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:58.607634 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dd363-8286-45b2-8b1a-2800923ad8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:58.607936 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dd363-8286-45b2-8b1a-2800923ad8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:58.610789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dd363-8286-45b2-8b1a-2800923ad8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:58.610789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dd363-8286-45b2-8b1a-2800923ad8fc admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:58.620580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1139/2259] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:58.624193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85970f18-e9e6-4798-93c8-fa98b6640a71 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:58.634176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85970f18-e9e6-4798-93c8-fa98b6640a71 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:58.638796 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab88428-9c2d-4e69-b72e-3bb846bbce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:58.639393 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88428-9c2d-4e69-b72e-3bb846bbce64 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:58.639668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88428-9c2d-4e69-b72e-3bb846bbce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:58.639969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab88428-9c2d-4e69-b72e-3bb846bbce64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:58.640612 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab88428-9c2d-4e69-b72e-3bb846bbce64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:58.642220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab88428-9c2d-4e69-b72e-3bb846bbce64 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:58.652648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1121/2260] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:58.656417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dd363-8286-45b2-8b1a-2800923ad8fc admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:58.666516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dd363-8286-45b2-8b1a-2800923ad8fc admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:58.671246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b068e-67c2-4077-a38d-3d6fa9081853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:58.671996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b068e-67c2-4077-a38d-3d6fa9081853 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:58.672278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b068e-67c2-4077-a38d-3d6fa9081853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:58.672553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b068e-67c2-4077-a38d-3d6fa9081853 admin admin] PATH_INFO: `/v3/projects/8680fb81cec84bc6b5daccc3aa2cf663/users/42e485efe5ee473b8afc65ac629a0bc2/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:58.687407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b068e-67c2-4077-a38d-3d6fa9081853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8680fb81cec84bc6b5daccc3aa2cf663, user_id=42e485efe5ee473b8afc65ac629a0bc2, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:58.690078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b068e-67c2-4077-a38d-3d6fa9081853 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:58.791807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1140/2261] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:58 2022] PUT /identity/v3/projects/8680fb81cec84bc6b5daccc3aa2cf663/users/42e485efe5ee473b8afc65ac629a0bc2/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:58.794993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2ed1-7233-4bd1-9145-4f9f9fb99071 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:58.795282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2ed1-7233-4bd1-9145-4f9f9fb99071 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:58.795555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2ed1-7233-4bd1-9145-4f9f9fb99071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:58.819745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:58.829655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7f8c2ed1-7233-4bd1-9145-4f9f9fb99071 None None] MFA Rules not processed for user `42e485efe5ee473b8afc65ac629a0bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:58.830896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7f8c2ed1-7233-4bd1-9145-4f9f9fb99071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:58.901005 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1122/2262] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:58.906275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b068e-67c2-4077-a38d-3d6fa9081853 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:59.038651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b068e-67c2-4077-a38d-3d6fa9081853 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:59.048774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87518463-3957-4993-b498-86f07de88f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:59.050320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87518463-3957-4993-b498-86f07de88f83 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:59.050666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87518463-3957-4993-b498-86f07de88f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:59.051257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87518463-3957-4993-b498-86f07de88f83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:59.052020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87518463-3957-4993-b498-86f07de88f83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:59.054037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87518463-3957-4993-b498-86f07de88f83 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:59.123032 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1141/2263] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:24:58 2022] POST /identity/v3/projects => generated 381 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:24:59.126918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c2ed1-7233-4bd1-9145-4f9f9fb99071 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:59.139806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c2ed1-7233-4bd1-9145-4f9f9fb99071 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:59.211407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e8b2f-81bd-497d-b5d6-8f430fecda19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:59.212203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e8b2f-81bd-497d-b5d6-8f430fecda19 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:59.214654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e8b2f-81bd-497d-b5d6-8f430fecda19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:59.214654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e8b2f-81bd-497d-b5d6-8f430fecda19 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:59.214654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e8b2f-81bd-497d-b5d6-8f430fecda19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:59.216030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e8b2f-81bd-497d-b5d6-8f430fecda19 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:59.302963 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1123/2264] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:24:59 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:24:59.304622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87518463-3957-4993-b498-86f07de88f83 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:59.318459 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87518463-3957-4993-b498-86f07de88f83 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:59.323805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acc889-fda6-4e67-b7b3-d0ddc6e3cb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:59.324493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acc889-fda6-4e67-b7b3-d0ddc6e3cb2c admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:59.324908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acc889-fda6-4e67-b7b3-d0ddc6e3cb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:59.325167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acc889-fda6-4e67-b7b3-d0ddc6e3cb2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:59.326192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acc889-fda6-4e67-b7b3-d0ddc6e3cb2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:59.327671 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acc889-fda6-4e67-b7b3-d0ddc6e3cb2c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:59.342910 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1142/2265] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:24:59.346411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e8b2f-81bd-497d-b5d6-8f430fecda19 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:59.357229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e8b2f-81bd-497d-b5d6-8f430fecda19 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:59.362499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771af59e-5b34-4b4e-8f56-c8d9a75b75ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:59.363155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771af59e-5b34-4b4e-8f56-c8d9a75b75ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:59.363432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771af59e-5b34-4b4e-8f56-c8d9a75b75ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:59.363708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771af59e-5b34-4b4e-8f56-c8d9a75b75ea admin admin] PATH_INFO: `/v3/projects/906645c3666740a6a58d634f6b6b764a/users/9d33bf0faf06426a8bd68abbea76cc3f/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:59.379090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771af59e-5b34-4b4e-8f56-c8d9a75b75ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=906645c3666740a6a58d634f6b6b764a, user_id=9d33bf0faf06426a8bd68abbea76cc3f, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:59.381703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771af59e-5b34-4b4e-8f56-c8d9a75b75ea admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:59.460139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1124/2266] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:59 2022] PUT /identity/v3/projects/906645c3666740a6a58d634f6b6b764a/users/9d33bf0faf06426a8bd68abbea76cc3f/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:24:59.464061 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acc889-fda6-4e67-b7b3-d0ddc6e3cb2c admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:59.475103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acc889-fda6-4e67-b7b3-d0ddc6e3cb2c admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:59.512327 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:59.530135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3eab52-7a0e-44a6-b83f-15a7abffbf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:59.530732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eab52-7a0e-44a6-b83f-15a7abffbf33 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:59.531011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eab52-7a0e-44a6-b83f-15a7abffbf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:59.531283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eab52-7a0e-44a6-b83f-15a7abffbf33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:59.531947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3eab52-7a0e-44a6-b83f-15a7abffbf33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:59.534205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3eab52-7a0e-44a6-b83f-15a7abffbf33 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:59.545240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1143/2267] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:24:59.549440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771af59e-5b34-4b4e-8f56-c8d9a75b75ea admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:59.559956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771af59e-5b34-4b4e-8f56-c8d9a75b75ea admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:59.565043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af619a9d-d1e9-4b62-8171-c06b8f1e3282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:59.565714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af619a9d-d1e9-4b62-8171-c06b8f1e3282 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:59.566019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af619a9d-d1e9-4b62-8171-c06b8f1e3282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:59.566299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af619a9d-d1e9-4b62-8171-c06b8f1e3282 admin admin] PATH_INFO: `/v3/projects/906645c3666740a6a58d634f6b6b764a/users/9d33bf0faf06426a8bd68abbea76cc3f/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:59.568351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af619a9d-d1e9-4b62-8171-c06b8f1e3282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=906645c3666740a6a58d634f6b6b764a, user_id=9d33bf0faf06426a8bd68abbea76cc3f, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:59.571087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af619a9d-d1e9-4b62-8171-c06b8f1e3282 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:59.582234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1125/2268] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:24:59 2022] PUT /identity/v3/projects/906645c3666740a6a58d634f6b6b764a/users/9d33bf0faf06426a8bd68abbea76cc3f/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:24:59.593512 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb118c6-d06d-4ec1-8132-586fafb33327 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:59.593798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb118c6-d06d-4ec1-8132-586fafb33327 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:59.594075 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb118c6-d06d-4ec1-8132-586fafb33327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:59.622434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-7cb118c6-d06d-4ec1-8132-586fafb33327 None None] MFA Rules not processed for user `9d33bf0faf06426a8bd68abbea76cc3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:24:59.623829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7cb118c6-d06d-4ec1-8132-586fafb33327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:59.694378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1144/2269] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:24:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:24:59.705078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af619a9d-d1e9-4b62-8171-c06b8f1e3282 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:24:59.707302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-af619a9d-d1e9-4b62-8171-c06b8f1e3282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:24:59.775387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af619a9d-d1e9-4b62-8171-c06b8f1e3282 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:24:59.778965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b213a7-7181-4eb1-a4ac-6dc4ec4df55b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:24:59.779565 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b213a7-7181-4eb1-a4ac-6dc4ec4df55b service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:24:59.779894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b213a7-7181-4eb1-a4ac-6dc4ec4df55b service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:24:59.780325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b213a7-7181-4eb1-a4ac-6dc4ec4df55b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:24:59.786698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:24:59.790355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b213a7-7181-4eb1-a4ac-6dc4ec4df55b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:24:59.791779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b213a7-7181-4eb1-a4ac-6dc4ec4df55b service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:24:59.801212 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1126/2270] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:24:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:00.465535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb118c6-d06d-4ec1-8132-586fafb33327 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:00.527381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb118c6-d06d-4ec1-8132-586fafb33327 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:00.532063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:00.532697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:00.532976 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:00.533873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] PATH_INFO: `/v3/users/42e485efe5ee473b8afc65ac629a0bc2` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:00.535019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] RBAC: Authorizing `identity:delete_user(user_id=42e485efe5ee473b8afc65ac629a0bc2)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:00.536974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:00.729685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42e485efe5ee473b8afc65ac629a0bc2'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:00.850243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42e485efe5ee473b8afc65ac629a0bc2'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:00.850860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42e485efe5ee473b8afc65ac629a0bc2'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:00.851519 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42e485efe5ee473b8afc65ac629a0bc2'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:00.859016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42e485efe5ee473b8afc65ac629a0bc2'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:00.860166 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:00.876855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1145/2271] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:25:00 2022] DELETE /identity/v3/users/42e485efe5ee473b8afc65ac629a0bc2 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:00.882020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021331bb-27c4-4bd4-892a-d7d93ee8cd27 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:00.882520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021331bb-27c4-4bd4-892a-d7d93ee8cd27 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:00.882624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021331bb-27c4-4bd4-892a-d7d93ee8cd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:00.903602 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-021331bb-27c4-4bd4-892a-d7d93ee8cd27 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:00.905486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-021331bb-27c4-4bd4-892a-d7d93ee8cd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:00.964939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1127/2272] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:25:00.974577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:00.975996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:01.035129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a502af6-f139-4beb-86ee-7cac2eb2c24b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:01.038669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae4a91-2e3d-4bb0-9d10-0044f5cef633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:01.039270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae4a91-2e3d-4bb0-9d10-0044f5cef633 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:01.039546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae4a91-2e3d-4bb0-9d10-0044f5cef633 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:01.039853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae4a91-2e3d-4bb0-9d10-0044f5cef633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:01.049552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae4a91-2e3d-4bb0-9d10-0044f5cef633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:01.053639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae4a91-2e3d-4bb0-9d10-0044f5cef633 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:01.061492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1146/2273] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:01.476361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021331bb-27c4-4bd4-892a-d7d93ee8cd27 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:01.476361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-021331bb-27c4-4bd4-892a-d7d93ee8cd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:01.491024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021331bb-27c4-4bd4-892a-d7d93ee8cd27 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:01.497942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:01.498882 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:01.499321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:01.500883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] PATH_INFO: `/v3/projects/8680fb81cec84bc6b5daccc3aa2cf663` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:01.503196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8680fb81cec84bc6b5daccc3aa2cf663)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:01.505633 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:01.643332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8680fb81cec84bc6b5daccc3aa2cf663', 'request_id': 'req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:01.741839 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8680fb81cec84bc6b5daccc3aa2cf663', 'request_id': 'req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:01.753736 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8680fb81cec84bc6b5daccc3aa2cf663', 'request_id': 'req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:01.764870 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] The token cache is being invalidate because project 8680fb81cec84bc6b5daccc3aa2cf663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:25:01.765165 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:01.769059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:01.769059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1128/2274] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:25:01 2022] DELETE /identity/v3/projects/8680fb81cec84bc6b5daccc3aa2cf663 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:01.772287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae4a91-2e3d-4bb0-9d10-0044f5cef633 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:01.782560 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae4a91-2e3d-4bb0-9d10-0044f5cef633 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:01.843245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e052b1-52c2-4a1d-a56f-a046baf078f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:01.843245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e052b1-52c2-4a1d-a56f-a046baf078f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:01.843245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e052b1-52c2-4a1d-a56f-a046baf078f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:01.843674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e052b1-52c2-4a1d-a56f-a046baf078f4 admin admin] PATH_INFO: `/v3/users/9d33bf0faf06426a8bd68abbea76cc3f` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:01.844484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e052b1-52c2-4a1d-a56f-a046baf078f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d33bf0faf06426a8bd68abbea76cc3f)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:01.847009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e052b1-52c2-4a1d-a56f-a046baf078f4 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:02.047919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-09e052b1-52c2-4a1d-a56f-a046baf078f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d33bf0faf06426a8bd68abbea76cc3f'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:02.096805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-09e052b1-52c2-4a1d-a56f-a046baf078f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d33bf0faf06426a8bd68abbea76cc3f'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:02.097198 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-09e052b1-52c2-4a1d-a56f-a046baf078f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d33bf0faf06426a8bd68abbea76cc3f'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:02.097874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-09e052b1-52c2-4a1d-a56f-a046baf078f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d33bf0faf06426a8bd68abbea76cc3f'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:02.105472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-09e052b1-52c2-4a1d-a56f-a046baf078f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d33bf0faf06426a8bd68abbea76cc3f'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:02.122598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1147/2275] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:25:01 2022] DELETE /identity/v3/users/9d33bf0faf06426a8bd68abbea76cc3f => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:02.509916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:02.511387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:02.525254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7b2f5-74c7-40e3-a5bb-117047b02d16 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:02.582534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:02.583163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:02.583439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:02.583710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] PATH_INFO: `/v3/projects/906645c3666740a6a58d634f6b6b764a` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:02.584997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=906645c3666740a6a58d634f6b6b764a)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:02.587039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:02.609954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:02.681336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '906645c3666740a6a58d634f6b6b764a', 'request_id': 'req-c15fd3af-c163-4fe7-bf42-31023de76d9f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:02.744456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '906645c3666740a6a58d634f6b6b764a', 'request_id': 'req-c15fd3af-c163-4fe7-bf42-31023de76d9f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:02.751543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '906645c3666740a6a58d634f6b6b764a', 'request_id': 'req-c15fd3af-c163-4fe7-bf42-31023de76d9f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:02.762654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] The token cache is being invalidate because project 906645c3666740a6a58d634f6b6b764a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:25:02.762944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:02.763592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c15fd3af-c163-4fe7-bf42-31023de76d9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:02.765117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1129/2276] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:25:02 2022] DELETE /identity/v3/projects/906645c3666740a6a58d634f6b6b764a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:02.770943 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf542aae-b327-4c8a-ad29-7eccb16bfec6 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:02.770943 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf542aae-b327-4c8a-ad29-7eccb16bfec6 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:02.771145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf542aae-b327-4c8a-ad29-7eccb16bfec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:02.779260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:02.794383 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-bf542aae-b327-4c8a-ad29-7eccb16bfec6 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:02.795631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-bf542aae-b327-4c8a-ad29-7eccb16bfec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:02.855875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1148/2277] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:02.864646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595237f0-3640-4567-b0cd-199b6eda315d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:02.868209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595237f0-3640-4567-b0cd-199b6eda315d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:02.868209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595237f0-3640-4567-b0cd-199b6eda315d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:02.886398 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-595237f0-3640-4567-b0cd-199b6eda315d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:02.887831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-595237f0-3640-4567-b0cd-199b6eda315d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:02.896150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1130/2278] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:02.905565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf542aae-b327-4c8a-ad29-7eccb16bfec6 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:02.915987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf542aae-b327-4c8a-ad29-7eccb16bfec6 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:02.920987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3631ef-bae4-4319-a575-097f151642f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:02.921598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3631ef-bae4-4319-a575-097f151642f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:02.922929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3631ef-bae4-4319-a575-097f151642f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:02.922929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3631ef-bae4-4319-a575-097f151642f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:02.922929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3631ef-bae4-4319-a575-097f151642f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:25:02.923248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3631ef-bae4-4319-a575-097f151642f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:02.924745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3631ef-bae4-4319-a575-097f151642f8 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:02.937045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1149/2279] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:25:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:25:02.941715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595237f0-3640-4567-b0cd-199b6eda315d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:02.951092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595237f0-3640-4567-b0cd-199b6eda315d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:02.955980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd28ca7-9f60-4a90-a0db-259ccfd107e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:02.956608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd28ca7-9f60-4a90-a0db-259ccfd107e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:02.956885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd28ca7-9f60-4a90-a0db-259ccfd107e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:02.957155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd28ca7-9f60-4a90-a0db-259ccfd107e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:02.957918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd28ca7-9f60-4a90-a0db-259ccfd107e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:02.959795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd28ca7-9f60-4a90-a0db-259ccfd107e6 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:03.069044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1131/2280] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:25:02 2022] POST /identity/v3/projects => generated 371 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:25:03.070206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3631ef-bae4-4319-a575-097f151642f8 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:03.071976 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9e3631ef-bae4-4319-a575-097f151642f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:03.072935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd28ca7-9f60-4a90-a0db-259ccfd107e6 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:03.086730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3631ef-bae4-4319-a575-097f151642f8 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:03.137434 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd28ca7-9f60-4a90-a0db-259ccfd107e6 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:03.142307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d60d9-c6a3-4baa-a938-0e85d0a0c1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:03.142912 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d60d9-c6a3-4baa-a938-0e85d0a0c1f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:03.143195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d60d9-c6a3-4baa-a938-0e85d0a0c1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:03.143480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d60d9-c6a3-4baa-a938-0e85d0a0c1f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:03.144751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d60d9-c6a3-4baa-a938-0e85d0a0c1f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:03.146706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d60d9-c6a3-4baa-a938-0e85d0a0c1f0 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:03.147889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c53a7-d14a-4081-a6a7-bee41b2fa86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:03.148557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c53a7-d14a-4081-a6a7-bee41b2fa86d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:03.148836 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c53a7-d14a-4081-a6a7-bee41b2fa86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:03.149111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c53a7-d14a-4081-a6a7-bee41b2fa86d admin admin] PATH_INFO: `/v3/users/f1d9452012f146758f81e78a104e7aba` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:03.150240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c53a7-d14a-4081-a6a7-bee41b2fa86d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1d9452012f146758f81e78a104e7aba)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:03.152238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c53a7-d14a-4081-a6a7-bee41b2fa86d admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:03.247482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1132/2281] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:25:03 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:25:03.251357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d60d9-c6a3-4baa-a938-0e85d0a0c1f0 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:03.260785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d60d9-c6a3-4baa-a938-0e85d0a0c1f0 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:03.265401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e915-ba57-411b-a22f-a6a95554ea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:03.265973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e915-ba57-411b-a22f-a6a95554ea91 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:03.266252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e915-ba57-411b-a22f-a6a95554ea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:03.266531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e915-ba57-411b-a22f-a6a95554ea91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:03.267159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1e915-ba57-411b-a22f-a6a95554ea91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:03.269385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1e915-ba57-411b-a22f-a6a95554ea91 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:03.282242 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1133/2282] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:25:03.283306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e915-ba57-411b-a22f-a6a95554ea91 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:03.293342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e915-ba57-411b-a22f-a6a95554ea91 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:03.298430 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e42d7-e5c6-494f-8cd9-6f2f2b83341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:03.299015 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e42d7-e5c6-494f-8cd9-6f2f2b83341e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:03.299327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e42d7-e5c6-494f-8cd9-6f2f2b83341e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:03.299650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e42d7-e5c6-494f-8cd9-6f2f2b83341e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:03.300300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e42d7-e5c6-494f-8cd9-6f2f2b83341e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:03.301945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e42d7-e5c6-494f-8cd9-6f2f2b83341e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:03.315945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1134/2283] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:25:03.317197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e42d7-e5c6-494f-8cd9-6f2f2b83341e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:03.327148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e42d7-e5c6-494f-8cd9-6f2f2b83341e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:03.332331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b75661-c08a-4aeb-bddf-accb981c97f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:03.332961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b75661-c08a-4aeb-bddf-accb981c97f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:03.333240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b75661-c08a-4aeb-bddf-accb981c97f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:03.333513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b75661-c08a-4aeb-bddf-accb981c97f8 admin admin] PATH_INFO: `/v3/projects/e8badc924cf541b5b87a87ed9dad845d/users/7a6f7c99c33641f394bada27d4aa28e6/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:03.349628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b75661-c08a-4aeb-bddf-accb981c97f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8badc924cf541b5b87a87ed9dad845d, user_id=7a6f7c99c33641f394bada27d4aa28e6, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:03.352334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b75661-c08a-4aeb-bddf-accb981c97f8 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:03.359608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-869c53a7-d14a-4081-a6a7-bee41b2fa86d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1d9452012f146758f81e78a104e7aba'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:03.443283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1135/2284] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:25:03 2022] PUT /identity/v3/projects/e8badc924cf541b5b87a87ed9dad845d/users/7a6f7c99c33641f394bada27d4aa28e6/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:25:03.447009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-869c53a7-d14a-4081-a6a7-bee41b2fa86d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1d9452012f146758f81e78a104e7aba'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:03.447753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-869c53a7-d14a-4081-a6a7-bee41b2fa86d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1d9452012f146758f81e78a104e7aba'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:03.448393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-869c53a7-d14a-4081-a6a7-bee41b2fa86d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1d9452012f146758f81e78a104e7aba'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:03.448853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c5163-aad5-401e-b03c-699293adfe3f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:03.449237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c5163-aad5-401e-b03c-699293adfe3f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:03.449577 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c5163-aad5-401e-b03c-699293adfe3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:03.457154 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-869c53a7-d14a-4081-a6a7-bee41b2fa86d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1d9452012f146758f81e78a104e7aba'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:03.468953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-2d7c5163-aad5-401e-b03c-699293adfe3f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:03.470163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2d7c5163-aad5-401e-b03c-699293adfe3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:03.477908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1150/2285] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:25:03 2022] DELETE /identity/v3/users/f1d9452012f146758f81e78a104e7aba => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:25:03.481191 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05d7c0-cb58-4005-a387-240d19c80939 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:03.481473 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05d7c0-cb58-4005-a387-240d19c80939 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:03.481747 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05d7c0-cb58-4005-a387-240d19c80939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:03.487861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:03.500756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-aa05d7c0-cb58-4005-a387-240d19c80939 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:03.501953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-aa05d7c0-cb58-4005-a387-240d19c80939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:03.506570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:03.531254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1136/2286] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:03.542179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c5163-aad5-401e-b03c-699293adfe3f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:03.545225 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2d7c5163-aad5-401e-b03c-699293adfe3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:03.562564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1151/2287] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:25:03.572649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05d7c0-cb58-4005-a387-240d19c80939 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:03.574051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-aa05d7c0-cb58-4005-a387-240d19c80939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:03.604776 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c5163-aad5-401e-b03c-699293adfe3f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:03.629178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05d7c0-cb58-4005-a387-240d19c80939 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:03.632829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b55a7-31f0-4331-b00a-bce46aa92fd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:03.633503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b55a7-31f0-4331-b00a-bce46aa92fd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:03.633788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b55a7-31f0-4331-b00a-bce46aa92fd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:03.634057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b55a7-31f0-4331-b00a-bce46aa92fd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:03.642934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b55a7-31f0-4331-b00a-bce46aa92fd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:03.645055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b55a7-31f0-4331-b00a-bce46aa92fd5 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:03.653239 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d242203-f7f8-4c00-8321-e87f92f58b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:03.653920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d242203-f7f8-4c00-8321-e87f92f58b9f service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:03.654311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d242203-f7f8-4c00-8321-e87f92f58b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:03.654594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d242203-f7f8-4c00-8321-e87f92f58b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:03.654832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1152/2288] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:25:03.664783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d242203-f7f8-4c00-8321-e87f92f58b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:03.666234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d242203-f7f8-4c00-8321-e87f92f58b9f service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:03.676301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1137/2289] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:25:04.110522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b55a7-31f0-4331-b00a-bce46aa92fd5 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:04.110522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c78b55a7-31f0-4331-b00a-bce46aa92fd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:04.125111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b55a7-31f0-4331-b00a-bce46aa92fd5 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:04.130994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:04.131720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:04.132331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:04.132673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] PATH_INFO: `/v3/projects/98dc17238e154072b41ce1de1b8c22a5` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:04.134021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98dc17238e154072b41ce1de1b8c22a5)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:04.136142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:04.265422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98dc17238e154072b41ce1de1b8c22a5', 'request_id': 'req-06267909-cb15-4d55-9ac4-ffbec523fb29'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:04.270833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98dc17238e154072b41ce1de1b8c22a5', 'request_id': 'req-06267909-cb15-4d55-9ac4-ffbec523fb29'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:04.381571 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98dc17238e154072b41ce1de1b8c22a5', 'request_id': 'req-06267909-cb15-4d55-9ac4-ffbec523fb29'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:04.394156 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] The token cache is being invalidate because project 98dc17238e154072b41ce1de1b8c22a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:25:04.394451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:04.395110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-06267909-cb15-4d55-9ac4-ffbec523fb29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:04.396956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1153/2290] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:25:04 2022] DELETE /identity/v3/projects/98dc17238e154072b41ce1de1b8c22a5 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:04.401668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7608d-241b-4086-a814-ceb7ea906c63 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:04.401957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7608d-241b-4086-a814-ceb7ea906c63 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:04.402236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7608d-241b-4086-a814-ceb7ea906c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:04.427501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0bc7608d-241b-4086-a814-ceb7ea906c63 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:04.428789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0bc7608d-241b-4086-a814-ceb7ea906c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:04.504016 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1138/2291] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:25:04.511648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef8aae-6cce-4c8c-aca6-0a1e5c8f5508 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:04.511956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef8aae-6cce-4c8c-aca6-0a1e5c8f5508 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:04.512233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef8aae-6cce-4c8c-aca6-0a1e5c8f5508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:04.531682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-aeef8aae-6cce-4c8c-aca6-0a1e5c8f5508 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:04.532926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-aeef8aae-6cce-4c8c-aca6-0a1e5c8f5508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:04.541698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1154/2292] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:04.546556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7608d-241b-4086-a814-ceb7ea906c63 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:04.556805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7608d-241b-4086-a814-ceb7ea906c63 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:04.561590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487ec87-3ce6-4a12-9235-d5e51fe6be92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:04.562187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487ec87-3ce6-4a12-9235-d5e51fe6be92 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:04.564049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487ec87-3ce6-4a12-9235-d5e51fe6be92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:04.564049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487ec87-3ce6-4a12-9235-d5e51fe6be92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:04.564049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487ec87-3ce6-4a12-9235-d5e51fe6be92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:25:04.564049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487ec87-3ce6-4a12-9235-d5e51fe6be92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:04.565329 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487ec87-3ce6-4a12-9235-d5e51fe6be92 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:04.577763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1139/2293] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:25:04.589501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef8aae-6cce-4c8c-aca6-0a1e5c8f5508 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:04.591984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef8aae-6cce-4c8c-aca6-0a1e5c8f5508 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:04.597195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc4631-a3b9-4791-a7d5-27e8afed13f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:04.597808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc4631-a3b9-4791-a7d5-27e8afed13f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:04.598214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc4631-a3b9-4791-a7d5-27e8afed13f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:04.598487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc4631-a3b9-4791-a7d5-27e8afed13f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:04.599127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc4631-a3b9-4791-a7d5-27e8afed13f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:04.601201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc4631-a3b9-4791-a7d5-27e8afed13f8 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:04.716942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1155/2294] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:25:04 2022] POST /identity/v3/projects => generated 397 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:25:04.721117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487ec87-3ce6-4a12-9235-d5e51fe6be92 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:04.731710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487ec87-3ce6-4a12-9235-d5e51fe6be92 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:04.736933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:04.789348 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5203a9-7a3c-44f7-aa40-9636d6495b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:04.789957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5203a9-7a3c-44f7-aa40-9636d6495b0d admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:04.790230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5203a9-7a3c-44f7-aa40-9636d6495b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:04.790499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5203a9-7a3c-44f7-aa40-9636d6495b0d admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:04.791131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5203a9-7a3c-44f7-aa40-9636d6495b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:04.793133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5203a9-7a3c-44f7-aa40-9636d6495b0d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:04.907587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1140/2295] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:25:04 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:25:04.912216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc4631-a3b9-4791-a7d5-27e8afed13f8 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:04.922986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc4631-a3b9-4791-a7d5-27e8afed13f8 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:04.927739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538aedc9-3514-492a-9402-34852abe21b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:04.928363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538aedc9-3514-492a-9402-34852abe21b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:04.928723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538aedc9-3514-492a-9402-34852abe21b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:04.929004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538aedc9-3514-492a-9402-34852abe21b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:04.929668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538aedc9-3514-492a-9402-34852abe21b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:04.931260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538aedc9-3514-492a-9402-34852abe21b6 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:04.932521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:04.941555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1156/2296] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:25:04.945301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5203a9-7a3c-44f7-aa40-9636d6495b0d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:04.955465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5203a9-7a3c-44f7-aa40-9636d6495b0d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:04.960641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649be5d-1305-42c3-94a2-c88504b9e34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:04.961233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649be5d-1305-42c3-94a2-c88504b9e34f admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:04.961514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649be5d-1305-42c3-94a2-c88504b9e34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:04.961795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649be5d-1305-42c3-94a2-c88504b9e34f admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:04.962431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649be5d-1305-42c3-94a2-c88504b9e34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:04.964050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649be5d-1305-42c3-94a2-c88504b9e34f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:04.975109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1141/2297] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:25:04.979226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538aedc9-3514-492a-9402-34852abe21b6 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:04.989535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538aedc9-3514-492a-9402-34852abe21b6 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:04.997466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc58867-32af-47b8-87d6-d8faa193e84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:04.997466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc58867-32af-47b8-87d6-d8faa193e84a admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:04.997466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc58867-32af-47b8-87d6-d8faa193e84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:04.997466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc58867-32af-47b8-87d6-d8faa193e84a admin admin] PATH_INFO: `/v3/projects/244b111cb0084cdf91497d3e148a698c/users/8920e2b191644edcbffdb5d559fb55ce/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:05.011819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc58867-32af-47b8-87d6-d8faa193e84a admin admin] RBAC: Authorizing `identity:create_grant(project_id=244b111cb0084cdf91497d3e148a698c, user_id=8920e2b191644edcbffdb5d559fb55ce, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:05.014492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc58867-32af-47b8-87d6-d8faa193e84a admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:05.090481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1157/2298] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:25:04 2022] PUT /identity/v3/projects/244b111cb0084cdf91497d3e148a698c/users/8920e2b191644edcbffdb5d559fb55ce/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:05.102520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f87894-5d7e-4e7e-b525-3aa370c9aa7c None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:05.102872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f87894-5d7e-4e7e-b525-3aa370c9aa7c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:05.103219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f87894-5d7e-4e7e-b525-3aa370c9aa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:05.145041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bab5e2-3d48-481f-9ab7-6686699dc145 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:05.153551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bab5e2-3d48-481f-9ab7-6686699dc145 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:05.153551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bab5e2-3d48-481f-9ab7-6686699dc145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:05.155536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-53f87894-5d7e-4e7e-b525-3aa370c9aa7c None None] MFA Rules not processed for user `8920e2b191644edcbffdb5d559fb55ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:05.157118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-53f87894-5d7e-4e7e-b525-3aa370c9aa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:05.185399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-d2bab5e2-3d48-481f-9ab7-6686699dc145 None None] MFA Rules not processed for user `7a6f7c99c33641f394bada27d4aa28e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:05.185399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d2bab5e2-3d48-481f-9ab7-6686699dc145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:05.264993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1142/2299] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:05 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:05.285763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f87894-5d7e-4e7e-b525-3aa370c9aa7c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:05.287805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-53f87894-5d7e-4e7e-b525-3aa370c9aa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:05.302838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1158/2300] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:25:05.306766 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bab5e2-3d48-481f-9ab7-6686699dc145 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:05.308205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d2bab5e2-3d48-481f-9ab7-6686699dc145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:05.351045 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f87894-5d7e-4e7e-b525-3aa370c9aa7c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:05.354709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bf468-f908-48b8-8cce-d992cad38385 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:05.355338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bf468-f908-48b8-8cce-d992cad38385 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:05.355617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bf468-f908-48b8-8cce-d992cad38385 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:05.355924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bf468-f908-48b8-8cce-d992cad38385 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:05.364348 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bf468-f908-48b8-8cce-d992cad38385 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:05.365869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3bf468-f908-48b8-8cce-d992cad38385 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:05.372528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bab5e2-3d48-481f-9ab7-6686699dc145 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:05.375656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1143/2301] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:05 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:25:05.377459 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf831fc0-f1ed-43c9-a7b7-8cbc78ddb5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:05.378591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf831fc0-f1ed-43c9-a7b7-8cbc78ddb5ec admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:05.378873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf831fc0-f1ed-43c9-a7b7-8cbc78ddb5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:05.379149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf831fc0-f1ed-43c9-a7b7-8cbc78ddb5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:05.379858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf831fc0-f1ed-43c9-a7b7-8cbc78ddb5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:05.381702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf831fc0-f1ed-43c9-a7b7-8cbc78ddb5ec admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:05.481134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1159/2302] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:25:05 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:25:05.485884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bf468-f908-48b8-8cce-d992cad38385 service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:05.502141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bf468-f908-48b8-8cce-d992cad38385 service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:05.524524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:05.573836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbd63c-b56d-4a29-885a-9c668d8df52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:05.574501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbd63c-b56d-4a29-885a-9c668d8df52c admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:05.574724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbd63c-b56d-4a29-885a-9c668d8df52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:05.575209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbd63c-b56d-4a29-885a-9c668d8df52c admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:05.575888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbd63c-b56d-4a29-885a-9c668d8df52c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:05.577885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbd63c-b56d-4a29-885a-9c668d8df52c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:05.660261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1144/2303] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:25:05 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:25:05.664176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf831fc0-f1ed-43c9-a7b7-8cbc78ddb5ec admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:05.675639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf831fc0-f1ed-43c9-a7b7-8cbc78ddb5ec admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:05.680553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f953d-b3ae-4d34-8341-221c1e09f542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:05.681042 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f953d-b3ae-4d34-8341-221c1e09f542 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:05.681317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f953d-b3ae-4d34-8341-221c1e09f542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:05.681587 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f953d-b3ae-4d34-8341-221c1e09f542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:05.682218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f953d-b3ae-4d34-8341-221c1e09f542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:05.683826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f953d-b3ae-4d34-8341-221c1e09f542 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:05.694155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1160/2304] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:25:05.698110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbd63c-b56d-4a29-885a-9c668d8df52c admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:05.708068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbd63c-b56d-4a29-885a-9c668d8df52c admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:05.713069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f716da-b625-438c-8ff8-af9b4fc3405b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:05.713726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f716da-b625-438c-8ff8-af9b4fc3405b admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:05.714008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f716da-b625-438c-8ff8-af9b4fc3405b admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:05.714344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f716da-b625-438c-8ff8-af9b4fc3405b admin admin] PATH_INFO: `/v3/projects/774ee7e2299b4a30ae78576e7298d87f/users/883c0e30cc864e0eaee292cbdf674c86/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:05.729216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f716da-b625-438c-8ff8-af9b4fc3405b admin admin] RBAC: Authorizing `identity:create_grant(project_id=774ee7e2299b4a30ae78576e7298d87f, user_id=883c0e30cc864e0eaee292cbdf674c86, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:05.731912 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f716da-b625-438c-8ff8-af9b4fc3405b admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:05.833540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1145/2305] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:25:05 2022] PUT /identity/v3/projects/774ee7e2299b4a30ae78576e7298d87f/users/883c0e30cc864e0eaee292cbdf674c86/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:05.837649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f953d-b3ae-4d34-8341-221c1e09f542 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:05.848299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f953d-b3ae-4d34-8341-221c1e09f542 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:05.881087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:05.905793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938aa0de-9f1f-407e-b616-3cb5c6551507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:05.906398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aa0de-9f1f-407e-b616-3cb5c6551507 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:05.909762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aa0de-9f1f-407e-b616-3cb5c6551507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:05.909762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aa0de-9f1f-407e-b616-3cb5c6551507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:05.909762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938aa0de-9f1f-407e-b616-3cb5c6551507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:05.909762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938aa0de-9f1f-407e-b616-3cb5c6551507 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:05.920753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1161/2306] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:25:05.925894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f716da-b625-438c-8ff8-af9b4fc3405b admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:05.937332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f716da-b625-438c-8ff8-af9b4fc3405b admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:05.942172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac24bd-38f8-43b8-bf79-d166729b5d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:05.942852 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac24bd-38f8-43b8-bf79-d166729b5d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:05.943131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac24bd-38f8-43b8-bf79-d166729b5d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:05.943412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac24bd-38f8-43b8-bf79-d166729b5d60 admin admin] PATH_INFO: `/v3/projects/774ee7e2299b4a30ae78576e7298d87f/users/883c0e30cc864e0eaee292cbdf674c86/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:05.945350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac24bd-38f8-43b8-bf79-d166729b5d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=774ee7e2299b4a30ae78576e7298d87f, user_id=883c0e30cc864e0eaee292cbdf674c86, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:05.948007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac24bd-38f8-43b8-bf79-d166729b5d60 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:05.958477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1146/2307] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:25:05 2022] PUT /identity/v3/projects/774ee7e2299b4a30ae78576e7298d87f/users/883c0e30cc864e0eaee292cbdf674c86/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:07.223338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175f1c-bce3-470a-b99e-c3b6f4c5c28c None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:07.223338 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175f1c-bce3-470a-b99e-c3b6f4c5c28c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:07.223822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175f1c-bce3-470a-b99e-c3b6f4c5c28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:07.257227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e1175f1c-bce3-470a-b99e-c3b6f4c5c28c None None] MFA Rules not processed for user `883c0e30cc864e0eaee292cbdf674c86`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:07.258461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e1175f1c-bce3-470a-b99e-c3b6f4c5c28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:07.359631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1162/2308] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:07.371508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac24bd-38f8-43b8-bf79-d166729b5d60 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:07.437367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac24bd-38f8-43b8-bf79-d166729b5d60 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:07.441447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc365e6-2121-415e-9784-0fe9bc89e7f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:07.442081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc365e6-2121-415e-9784-0fe9bc89e7f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:07.442326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc365e6-2121-415e-9784-0fe9bc89e7f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:07.442595 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc365e6-2121-415e-9784-0fe9bc89e7f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:07.515773 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc365e6-2121-415e-9784-0fe9bc89e7f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:07.517249 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc365e6-2121-415e-9784-0fe9bc89e7f6 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:07.527012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1147/2309] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:10.348610 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1175f1c-bce3-470a-b99e-c3b6f4c5c28c None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:10.359060 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1175f1c-bce3-470a-b99e-c3b6f4c5c28c None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:10.363628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aad5e7-b150-43bb-8dad-6fdf3a294e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:10.364268 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aad5e7-b150-43bb-8dad-6fdf3a294e84 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:10.364548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aad5e7-b150-43bb-8dad-6fdf3a294e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:10.364821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aad5e7-b150-43bb-8dad-6fdf3a294e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:10.373909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aad5e7-b150-43bb-8dad-6fdf3a294e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:10.375365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aad5e7-b150-43bb-8dad-6fdf3a294e84 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:10.385356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1163/2310] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:17.522952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:20.380934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=53 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:21.795043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c36d4-d693-47cb-af14-1a583f85e9a3 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:21.796004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c36d4-d693-47cb-af14-1a583f85e9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:21.796004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c36d4-d693-47cb-af14-1a583f85e9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:21.815846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-1b8c36d4-d693-47cb-af14-1a583f85e9a3 None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:21.816974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1b8c36d4-d693-47cb-af14-1a583f85e9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:21.878121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1148/2311] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:25:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:21.934499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aad5e7-b150-43bb-8dad-6fdf3a294e84 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:21.936648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b4aad5e7-b150-43bb-8dad-6fdf3a294e84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:22.023391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aad5e7-b150-43bb-8dad-6fdf3a294e84 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:22.028898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ba61f-b5eb-4e74-84fb-af96ed930527 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:22.032379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ba61f-b5eb-4e74-84fb-af96ed930527 service swift] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:22.032379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ba61f-b5eb-4e74-84fb-af96ed930527 service swift] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:22.032379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ba61f-b5eb-4e74-84fb-af96ed930527 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:22.038403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ba61f-b5eb-4e74-84fb-af96ed930527 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:22.040039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ba61f-b5eb-4e74-84fb-af96ed930527 service swift] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:22.049012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1164/2312] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:25:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:25:22.669220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c36d4-d693-47cb-af14-1a583f85e9a3 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:22.779319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c36d4-d693-47cb-af14-1a583f85e9a3 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:22.796078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:22.796796 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:22.797020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:22.797300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] PATH_INFO: `/v3/users/8920e2b191644edcbffdb5d559fb55ce` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:22.798790 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8920e2b191644edcbffdb5d559fb55ce)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:22.800755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:23.052195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8920e2b191644edcbffdb5d559fb55ce'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:23.245509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8920e2b191644edcbffdb5d559fb55ce'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:23.245509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8920e2b191644edcbffdb5d559fb55ce'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:23.245509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8920e2b191644edcbffdb5d559fb55ce'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:23.245509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8920e2b191644edcbffdb5d559fb55ce'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:23.245509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1149/2313] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:25:22 2022] DELETE /identity/v3/users/8920e2b191644edcbffdb5d559fb55ce => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:23.245944 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f016708-23e6-4d8a-ab48-585d6a0b71bd None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:23.245944 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f016708-23e6-4d8a-ab48-585d6a0b71bd None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:23.245944 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f016708-23e6-4d8a-ab48-585d6a0b71bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:23.317900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-0f016708-23e6-4d8a-ab48-585d6a0b71bd None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:23.339506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0f016708-23e6-4d8a-ab48-585d6a0b71bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:23.413324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1165/2314] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:23.423352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:23.425385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:23.459646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:23.493145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeadd26-5f85-4704-aba4-ab03fd044d69 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:23.497179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2166af-12eb-46f3-98ce-8f4d4e367775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:23.497751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2166af-12eb-46f3-98ce-8f4d4e367775 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:23.498061 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2166af-12eb-46f3-98ce-8f4d4e367775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:23.498306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2166af-12eb-46f3-98ce-8f4d4e367775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:23.508284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2166af-12eb-46f3-98ce-8f4d4e367775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:23.509704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2166af-12eb-46f3-98ce-8f4d4e367775 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:23.519456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1150/2315] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:23.937151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f016708-23e6-4d8a-ab48-585d6a0b71bd None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:23.938585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0f016708-23e6-4d8a-ab48-585d6a0b71bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:23.952706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f016708-23e6-4d8a-ab48-585d6a0b71bd None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:23.959793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:23.960811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:23.961094 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:23.961367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] PATH_INFO: `/v3/projects/244b111cb0084cdf91497d3e148a698c` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:23.962583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=244b111cb0084cdf91497d3e148a698c)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:23.964601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:24.083614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '244b111cb0084cdf91497d3e148a698c', 'request_id': 'req-0f1d2d79-1481-4541-a1e5-5e35db3938a5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:24.090376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '244b111cb0084cdf91497d3e148a698c', 'request_id': 'req-0f1d2d79-1481-4541-a1e5-5e35db3938a5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:24.189391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '244b111cb0084cdf91497d3e148a698c', 'request_id': 'req-0f1d2d79-1481-4541-a1e5-5e35db3938a5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:24.207549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] The token cache is being invalidate because project 244b111cb0084cdf91497d3e148a698c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:25:24.207925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:24.208700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0f1d2d79-1481-4541-a1e5-5e35db3938a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:24.215486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1166/2316] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:25:23 2022] DELETE /identity/v3/projects/244b111cb0084cdf91497d3e148a698c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:24.216874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff62be-6057-4126-9028-63d9c4b04b07 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:24.223457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff62be-6057-4126-9028-63d9c4b04b07 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:24.223457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff62be-6057-4126-9028-63d9c4b04b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:24.255949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-98ff62be-6057-4126-9028-63d9c4b04b07 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:24.257231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-98ff62be-6057-4126-9028-63d9c4b04b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:24.351365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1151/2317] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:24.352878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b374d75-6c61-4a18-9303-483bd553a8b2 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:24.353415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b374d75-6c61-4a18-9303-483bd553a8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:24.353415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b374d75-6c61-4a18-9303-483bd553a8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:24.376847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8b374d75-6c61-4a18-9303-483bd553a8b2 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:24.378245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8b374d75-6c61-4a18-9303-483bd553a8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:24.387019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1167/2318] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:24.391980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff62be-6057-4126-9028-63d9c4b04b07 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:24.405993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff62be-6057-4126-9028-63d9c4b04b07 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:24.411065 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d34c4-774b-4b24-8af4-80b2398b12ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:24.413115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d34c4-774b-4b24-8af4-80b2398b12ea admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:24.413427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d34c4-774b-4b24-8af4-80b2398b12ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:24.413706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d34c4-774b-4b24-8af4-80b2398b12ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:24.414414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d34c4-774b-4b24-8af4-80b2398b12ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:25:24.414727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d34c4-774b-4b24-8af4-80b2398b12ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:24.416396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d34c4-774b-4b24-8af4-80b2398b12ea admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:24.429882 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1152/2319] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:25:24.433955 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b374d75-6c61-4a18-9303-483bd553a8b2 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:24.448677 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b374d75-6c61-4a18-9303-483bd553a8b2 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:24.458209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962decf-67d5-47e6-8c2e-a21c2bceb2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:24.458209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962decf-67d5-47e6-8c2e-a21c2bceb2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:24.458209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962decf-67d5-47e6-8c2e-a21c2bceb2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:24.458209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962decf-67d5-47e6-8c2e-a21c2bceb2b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:24.458645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962decf-67d5-47e6-8c2e-a21c2bceb2b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:24.459797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0962decf-67d5-47e6-8c2e-a21c2bceb2b9 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:24.464735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:24.554392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1168/2320] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:25:24 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:25:24.558079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d34c4-774b-4b24-8af4-80b2398b12ea admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:24.571450 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d34c4-774b-4b24-8af4-80b2398b12ea admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:24.634685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc08d68-5014-4a20-bc57-9c12d079d46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:24.635319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc08d68-5014-4a20-bc57-9c12d079d46a admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:24.635596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc08d68-5014-4a20-bc57-9c12d079d46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:24.635898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc08d68-5014-4a20-bc57-9c12d079d46a admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:24.636539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc08d68-5014-4a20-bc57-9c12d079d46a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:24.638574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc08d68-5014-4a20-bc57-9c12d079d46a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:24.716625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1153/2321] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:25:24 2022] POST /identity/v3/users => generated 288 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:25:24.720715 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962decf-67d5-47e6-8c2e-a21c2bceb2b9 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:24.731133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962decf-67d5-47e6-8c2e-a21c2bceb2b9 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:24.735953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038903c-05fa-40af-a73c-95d92cd40967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:24.736556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038903c-05fa-40af-a73c-95d92cd40967 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:24.736835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038903c-05fa-40af-a73c-95d92cd40967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:24.737158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038903c-05fa-40af-a73c-95d92cd40967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:24.737808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038903c-05fa-40af-a73c-95d92cd40967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:24.739431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038903c-05fa-40af-a73c-95d92cd40967 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:24.749502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1169/2322] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:25:24.753478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc08d68-5014-4a20-bc57-9c12d079d46a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:24.764118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc08d68-5014-4a20-bc57-9c12d079d46a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:24.769023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463f68f-61e9-4782-887d-4f9a251e2518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:24.770426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463f68f-61e9-4782-887d-4f9a251e2518 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:24.770426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463f68f-61e9-4782-887d-4f9a251e2518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:24.770426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463f68f-61e9-4782-887d-4f9a251e2518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:24.770808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463f68f-61e9-4782-887d-4f9a251e2518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:24.773062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463f68f-61e9-4782-887d-4f9a251e2518 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:24.783616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1154/2323] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:25:24.787654 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038903c-05fa-40af-a73c-95d92cd40967 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:24.797792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038903c-05fa-40af-a73c-95d92cd40967 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:24.802398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4b555-c0aa-494a-8602-4586ac12180e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:24.804907 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b555-c0aa-494a-8602-4586ac12180e admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:24.804907 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b555-c0aa-494a-8602-4586ac12180e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:24.804907 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4b555-c0aa-494a-8602-4586ac12180e admin admin] PATH_INFO: `/v3/projects/1ccb39d9f1db4220ac101df9cb4572cf/users/be10df0a4d1d42208c46615d6ddd970c/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:24.819114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4b555-c0aa-494a-8602-4586ac12180e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccb39d9f1db4220ac101df9cb4572cf, user_id=be10df0a4d1d42208c46615d6ddd970c, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:24.821983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4b555-c0aa-494a-8602-4586ac12180e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:24.889335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1170/2324] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:25:24 2022] PUT /identity/v3/projects/1ccb39d9f1db4220ac101df9cb4572cf/users/be10df0a4d1d42208c46615d6ddd970c/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:24.894495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807dd18-f1f4-48c4-905a-dd0e05cdb60f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:24.894883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807dd18-f1f4-48c4-905a-dd0e05cdb60f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:24.895164 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807dd18-f1f4-48c4-905a-dd0e05cdb60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:24.913845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c807dd18-f1f4-48c4-905a-dd0e05cdb60f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:24.915040 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c807dd18-f1f4-48c4-905a-dd0e05cdb60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:24.945209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:24.974036 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1155/2325] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:25:24.986168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4b555-c0aa-494a-8602-4586ac12180e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:24.986168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a6b4b555-c0aa-494a-8602-4586ac12180e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:25.042952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4b555-c0aa-494a-8602-4586ac12180e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:25.046812 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf1d65-76e6-4027-8917-26f25171170b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:25.047419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf1d65-76e6-4027-8917-26f25171170b service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:25.047703 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf1d65-76e6-4027-8917-26f25171170b service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:25.048009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf1d65-76e6-4027-8917-26f25171170b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:25.057412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf1d65-76e6-4027-8917-26f25171170b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:25.058856 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf1d65-76e6-4027-8917-26f25171170b service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:25.071743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1171/2326] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:28.749370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b88a-3d70-43dd-98e0-1c0563231c9d None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:28.749868 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b88a-3d70-43dd-98e0-1c0563231c9d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:28.749975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b88a-3d70-43dd-98e0-1c0563231c9d None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:28.751650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1156/2327] 10.222.0.15 () {54 vars in 961 bytes} [Wed Sep 7 16:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:25:28.756900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e8642-b281-47df-9666-11ae0becefc4 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:28.757409 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e8642-b281-47df-9666-11ae0becefc4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:28.757501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e8642-b281-47df-9666-11ae0becefc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:28.791914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-4b9e8642-b281-47df-9666-11ae0becefc4 None None] MFA Rules not processed for user `5fdfb0bf09db406ab5c09fbef887cbab`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:28.793190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4b9e8642-b281-47df-9666-11ae0becefc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:28.902613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b88a-3d70-43dd-98e0-1c0563231c9d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:28.905608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1172/2328] 10.222.0.15 () {58 vars in 1058 bytes} [Wed Sep 7 16:25:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:28.950532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b88a-3d70-43dd-98e0-1c0563231c9d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:28.973248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dda3e-a6d1-45ab-9851-0dbb01b54394 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:28.974216 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dda3e-a6d1-45ab-9851-0dbb01b54394 service placement] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:28.974797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dda3e-a6d1-45ab-9851-0dbb01b54394 service placement] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:28.975233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dda3e-a6d1-45ab-9851-0dbb01b54394 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:28.977561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-697dda3e-a6d1-45ab-9851-0dbb01b54394 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:28.994628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697dda3e-a6d1-45ab-9851-0dbb01b54394 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:28.996823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697dda3e-a6d1-45ab-9851-0dbb01b54394 service placement] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:29.048699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1157/2329] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:32.520421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc1c6c-5454-49da-a77c-4c7964db64fe None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:32.521591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc1c6c-5454-49da-a77c-4c7964db64fe None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:32.521591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc1c6c-5454-49da-a77c-4c7964db64fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:32.611154 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-d0fc1c6c-5454-49da-a77c-4c7964db64fe None None] MFA Rules not processed for user `be10df0a4d1d42208c46615d6ddd970c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:32.612625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d0fc1c6c-5454-49da-a77c-4c7964db64fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:32.674475 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:32.710310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1173/2330] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:32.713540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dda3e-a6d1-45ab-9851-0dbb01b54394 service placement] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:32.724544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dda3e-a6d1-45ab-9851-0dbb01b54394 service placement] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:32.731220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc4720-7910-49fb-b700-a90c65728bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:32.733752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc4720-7910-49fb-b700-a90c65728bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:32.733752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc4720-7910-49fb-b700-a90c65728bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:32.733752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc4720-7910-49fb-b700-a90c65728bc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:32.733752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc4720-7910-49fb-b700-a90c65728bc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:32.734938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc4720-7910-49fb-b700-a90c65728bc5 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:32.875468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1158/2331] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:25:32 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:25:32.876255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc1c6c-5454-49da-a77c-4c7964db64fe None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:32.897914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc1c6c-5454-49da-a77c-4c7964db64fe None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:32.999273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cae299-aefc-4654-9b3c-6233f63f570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:33.002091 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cae299-aefc-4654-9b3c-6233f63f570c admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:33.002381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cae299-aefc-4654-9b3c-6233f63f570c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:33.002801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cae299-aefc-4654-9b3c-6233f63f570c admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:33.003488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cae299-aefc-4654-9b3c-6233f63f570c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:33.009386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cae299-aefc-4654-9b3c-6233f63f570c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:33.141618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1174/2332] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:25:32 2022] POST /identity/v3/users => generated 296 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:25:33.153500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc4720-7910-49fb-b700-a90c65728bc5 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:33.201567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc4720-7910-49fb-b700-a90c65728bc5 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:33.216554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4616efd-fef4-4149-9628-8d9f964876f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:33.217506 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4616efd-fef4-4149-9628-8d9f964876f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:33.218036 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4616efd-fef4-4149-9628-8d9f964876f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:33.218573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4616efd-fef4-4149-9628-8d9f964876f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:33.219480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4616efd-fef4-4149-9628-8d9f964876f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:33.222026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4616efd-fef4-4149-9628-8d9f964876f8 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:33.251965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1159/2333] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:25:33.257686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cae299-aefc-4654-9b3c-6233f63f570c admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:33.275814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cae299-aefc-4654-9b3c-6233f63f570c admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:33.300943 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a747f38-bce2-4b62-aec9-c24a65ecedfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:33.301253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a747f38-bce2-4b62-aec9-c24a65ecedfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:33.301996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a747f38-bce2-4b62-aec9-c24a65ecedfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:33.303040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a747f38-bce2-4b62-aec9-c24a65ecedfc admin admin] PATH_INFO: `/v3/projects/dc17de2d8943481dbccd43109bcb3e85/users/6904372579ae456c852f70b903c6603e/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:33.326375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a747f38-bce2-4b62-aec9-c24a65ecedfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc17de2d8943481dbccd43109bcb3e85, user_id=6904372579ae456c852f70b903c6603e, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:33.331950 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a747f38-bce2-4b62-aec9-c24a65ecedfc admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:33.447576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1175/2334] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:25:33 2022] PUT /identity/v3/projects/dc17de2d8943481dbccd43109bcb3e85/users/6904372579ae456c852f70b903c6603e/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:33.449245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4616efd-fef4-4149-9628-8d9f964876f8 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:33.464441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4616efd-fef4-4149-9628-8d9f964876f8 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:33.593336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b3116-34d2-4f43-a62a-a7c4a99eed96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:33.594382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b3116-34d2-4f43-a62a-a7c4a99eed96 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:33.594382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b3116-34d2-4f43-a62a-a7c4a99eed96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:33.597979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b3116-34d2-4f43-a62a-a7c4a99eed96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:33.598731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b3116-34d2-4f43-a62a-a7c4a99eed96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:33.600568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b3116-34d2-4f43-a62a-a7c4a99eed96 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:33.629987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a747f38-bce2-4b62-aec9-c24a65ecedfc admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:33.632499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1160/2335] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:25:33.664282 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a747f38-bce2-4b62-aec9-c24a65ecedfc admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:33.681955 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d0191-1cbe-4025-82f4-8aff1b727582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:33.688810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d0191-1cbe-4025-82f4-8aff1b727582 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:33.689038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d0191-1cbe-4025-82f4-8aff1b727582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:33.689223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d0191-1cbe-4025-82f4-8aff1b727582 admin admin] PATH_INFO: `/v3/projects/dc17de2d8943481dbccd43109bcb3e85/users/6904372579ae456c852f70b903c6603e/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:33.692593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d0191-1cbe-4025-82f4-8aff1b727582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc17de2d8943481dbccd43109bcb3e85, user_id=6904372579ae456c852f70b903c6603e, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:33.696797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d0191-1cbe-4025-82f4-8aff1b727582 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:33.715135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1176/2336] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:25:33 2022] PUT /identity/v3/projects/dc17de2d8943481dbccd43109bcb3e85/users/6904372579ae456c852f70b903c6603e/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:36.400056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b3116-34d2-4f43-a62a-a7c4a99eed96 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:36.402918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e26b3116-34d2-4f43-a62a-a7c4a99eed96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:36.432344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b3116-34d2-4f43-a62a-a7c4a99eed96 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:36.571133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:36.572281 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:36.572635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:36.573049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] PATH_INFO: `/v3/users/afe1f2aeb4eb4f31a91e031d3a0f3a9f` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:36.576559 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe1f2aeb4eb4f31a91e031d3a0f3a9f)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:36.585592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:36.750236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:36.903052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe1f2aeb4eb4f31a91e031d3a0f3a9f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:37.092540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe1f2aeb4eb4f31a91e031d3a0f3a9f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:37.092540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe1f2aeb4eb4f31a91e031d3a0f3a9f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:37.092540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe1f2aeb4eb4f31a91e031d3a0f3a9f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:37.127824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe1f2aeb4eb4f31a91e031d3a0f3a9f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:37.197518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1161/2337] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:25:36 2022] DELETE /identity/v3/users/afe1f2aeb4eb4f31a91e031d3a0f3a9f => generated 0 bytes in 798 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:37.202133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112b415-7b6a-4bf5-843a-4b4cc5062f74 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:37.202655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112b415-7b6a-4bf5-843a-4b4cc5062f74 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:37.202743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112b415-7b6a-4bf5-843a-4b4cc5062f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:37.269002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-d112b415-7b6a-4bf5-843a-4b4cc5062f74 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:37.270596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d112b415-7b6a-4bf5-843a-4b4cc5062f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:37.418870 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1177/2338] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:37.429781 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:37.431299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:37.508961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80fcdca-7465-4d67-87de-f8436441aa66 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:37.518873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1528a96-336a-4179-aef3-f0da47879a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:37.521333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1528a96-336a-4179-aef3-f0da47879a82 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:37.522213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1528a96-336a-4179-aef3-f0da47879a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:37.522991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1528a96-336a-4179-aef3-f0da47879a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:37.536814 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1528a96-336a-4179-aef3-f0da47879a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:37.538635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1528a96-336a-4179-aef3-f0da47879a82 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:37.560439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1162/2339] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:38.404966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112b415-7b6a-4bf5-843a-4b4cc5062f74 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:38.410054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d112b415-7b6a-4bf5-843a-4b4cc5062f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:38.433495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112b415-7b6a-4bf5-843a-4b4cc5062f74 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:38.440702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:38.445487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:38.445893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:38.451266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] PATH_INFO: `/v3/projects/3811dcff000c4821bd1d1a2715d1fd21` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:38.455317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3811dcff000c4821bd1d1a2715d1fd21)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:38.464740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:38.628021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:38.660155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3811dcff000c4821bd1d1a2715d1fd21', 'request_id': 'req-f907831b-a167-41bf-9674-5c8aee178c99'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:38.669686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3811dcff000c4821bd1d1a2715d1fd21', 'request_id': 'req-f907831b-a167-41bf-9674-5c8aee178c99'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:38.746582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3811dcff000c4821bd1d1a2715d1fd21', 'request_id': 'req-f907831b-a167-41bf-9674-5c8aee178c99'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:38.788796 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] The token cache is being invalidate because project 3811dcff000c4821bd1d1a2715d1fd21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:25:38.790452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:38.799047 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:38.816987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1178/2340] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:25:38 2022] DELETE /identity/v3/projects/3811dcff000c4821bd1d1a2715d1fd21 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:42.465388 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4eeb9-eed8-4361-8ca8-00ebabd8f2ea None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:42.465946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4eeb9-eed8-4361-8ca8-00ebabd8f2ea None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:42.466147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4eeb9-eed8-4361-8ca8-00ebabd8f2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:42.517261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-2cb4eeb9-eed8-4361-8ca8-00ebabd8f2ea None None] MFA Rules not processed for user `6904372579ae456c852f70b903c6603e`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:42.519419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2cb4eeb9-eed8-4361-8ca8-00ebabd8f2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:42.686574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1163/2341] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:42 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:42.696270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:42.698506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:42.809161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907831b-a167-41bf-9674-5c8aee178c99 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:42.826690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353737e-bada-4d8f-8e2f-ec24d94e21a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:42.828852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353737e-bada-4d8f-8e2f-ec24d94e21a0 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:42.829297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353737e-bada-4d8f-8e2f-ec24d94e21a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:42.829719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353737e-bada-4d8f-8e2f-ec24d94e21a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:42.831906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e353737e-bada-4d8f-8e2f-ec24d94e21a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:42.946773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353737e-bada-4d8f-8e2f-ec24d94e21a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:42.950017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353737e-bada-4d8f-8e2f-ec24d94e21a0 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:42.962352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1179/2342] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:46.099327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bbe8d-5903-4aac-b227-3b8c366f595c None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:46.100573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bbe8d-5903-4aac-b227-3b8c366f595c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:46.101231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bbe8d-5903-4aac-b227-3b8c366f595c None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:46.102799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1164/2343] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:25:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:25:46.114166 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552798b6-9cd6-493c-8d73-9aafef039e29 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:46.114779 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552798b6-9cd6-493c-8d73-9aafef039e29 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:46.114779 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552798b6-9cd6-493c-8d73-9aafef039e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:46.139173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-552798b6-9cd6-493c-8d73-9aafef039e29 None None] MFA Rules not processed for user `5fdfb0bf09db406ab5c09fbef887cbab`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:46.140616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-552798b6-9cd6-493c-8d73-9aafef039e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:46.235068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:46.257909 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1180/2344] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:25:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:46.285346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bbe8d-5903-4aac-b227-3b8c366f595c None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:46.297716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d01bbe8d-5903-4aac-b227-3b8c366f595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:46.365731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bbe8d-5903-4aac-b227-3b8c366f595c None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:46.378151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33910b99-4981-4d9a-9a49-5435dec2d0fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:46.378842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33910b99-4981-4d9a-9a49-5435dec2d0fe service placement] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:46.379122 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33910b99-4981-4d9a-9a49-5435dec2d0fe service placement] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:46.380172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33910b99-4981-4d9a-9a49-5435dec2d0fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:46.392912 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33910b99-4981-4d9a-9a49-5435dec2d0fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:46.394516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33910b99-4981-4d9a-9a49-5435dec2d0fe service placement] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:46.408793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1165/2345] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:47.354194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552798b6-9cd6-493c-8d73-9aafef039e29 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:47.356317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-552798b6-9cd6-493c-8d73-9aafef039e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:47.453486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552798b6-9cd6-493c-8d73-9aafef039e29 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:47.458817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:47.460039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:47.460668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:47.461140 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] PATH_INFO: `/v3/users/7a6f7c99c33641f394bada27d4aa28e6` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:47.462505 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6f7c99c33641f394bada27d4aa28e6)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:47.464680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:47.725339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6f7c99c33641f394bada27d4aa28e6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:47.789879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6f7c99c33641f394bada27d4aa28e6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:47.792335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6f7c99c33641f394bada27d4aa28e6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:47.793550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6f7c99c33641f394bada27d4aa28e6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:47.803832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6f7c99c33641f394bada27d4aa28e6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:47.823160 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1181/2346] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:25:47 2022] DELETE /identity/v3/users/7a6f7c99c33641f394bada27d4aa28e6 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:48.199300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33910b99-4981-4d9a-9a49-5435dec2d0fe service placement] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:48.213210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33910b99-4981-4d9a-9a49-5435dec2d0fe service placement] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:48.233691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:48.280957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:48.281674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:48.281957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:48.282230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] PATH_INFO: `/v3/projects/e8badc924cf541b5b87a87ed9dad845d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:48.283374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8badc924cf541b5b87a87ed9dad845d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:48.285315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:48.386072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8badc924cf541b5b87a87ed9dad845d', 'request_id': 'req-d74144c4-fcb2-437f-87c9-aa39589e53b9'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:48.442534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8badc924cf541b5b87a87ed9dad845d', 'request_id': 'req-d74144c4-fcb2-437f-87c9-aa39589e53b9'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:48.449181 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8badc924cf541b5b87a87ed9dad845d', 'request_id': 'req-d74144c4-fcb2-437f-87c9-aa39589e53b9'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:48.460094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] The token cache is being invalidate because project e8badc924cf541b5b87a87ed9dad845d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:25:48.464990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:48.464990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d74144c4-fcb2-437f-87c9-aa39589e53b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:48.464990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1166/2347] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:25:48 2022] DELETE /identity/v3/projects/e8badc924cf541b5b87a87ed9dad845d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:48.466420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:48.467855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:48.481413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b54ec54-9633-4dc3-9bc5-a07ae806bb36 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:48.538188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:48.539033 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:48.539320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:48.539597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] PATH_INFO: `/v3/users/883c0e30cc864e0eaee292cbdf674c86` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:48.540773 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=883c0e30cc864e0eaee292cbdf674c86)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:48.542698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:48.549748 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:48.729718 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '883c0e30cc864e0eaee292cbdf674c86'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:48.826933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '883c0e30cc864e0eaee292cbdf674c86'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:48.827599 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '883c0e30cc864e0eaee292cbdf674c86'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:48.828526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '883c0e30cc864e0eaee292cbdf674c86'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:48.839919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '883c0e30cc864e0eaee292cbdf674c86'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:48.842489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c962b-ed0d-4f7c-82ea-b8ec146dcd15 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:48.843002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c962b-ed0d-4f7c-82ea-b8ec146dcd15 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:48.843352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c962b-ed0d-4f7c-82ea-b8ec146dcd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:48.860184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1182/2348] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:25:48 2022] DELETE /identity/v3/users/883c0e30cc864e0eaee292cbdf674c86 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:48.873885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-f48c962b-ed0d-4f7c-82ea-b8ec146dcd15 None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:48.875133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f48c962b-ed0d-4f7c-82ea-b8ec146dcd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:48.939147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1167/2349] 10.222.0.15 () {58 vars in 967 bytes} [Wed Sep 7 16:25:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:48.953755 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:48.955253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:49.035049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975e517-8295-4905-9038-f98083676be8 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:49.040731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9346a8-b4db-4ee4-b8ee-b01dc6b50745 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:49.041341 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9346a8-b4db-4ee4-b8ee-b01dc6b50745 service swift] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:49.041624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9346a8-b4db-4ee4-b8ee-b01dc6b50745 service swift] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:49.041931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9346a8-b4db-4ee4-b8ee-b01dc6b50745 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:49.050625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9346a8-b4db-4ee4-b8ee-b01dc6b50745 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:49.052342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9346a8-b4db-4ee4-b8ee-b01dc6b50745 service swift] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:49.061327 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1183/2350] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:25:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:25:49.139224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c962b-ed0d-4f7c-82ea-b8ec146dcd15 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:49.141217 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f48c962b-ed0d-4f7c-82ea-b8ec146dcd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:49.199845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:49.214921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c962b-ed0d-4f7c-82ea-b8ec146dcd15 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:49.219855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:49.220717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:49.221009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:49.221443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] PATH_INFO: `/v3/projects/774ee7e2299b4a30ae78576e7298d87f` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:49.222710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=774ee7e2299b4a30ae78576e7298d87f)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:49.224668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:49.351207 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774ee7e2299b4a30ae78576e7298d87f', 'request_id': 'req-84115975-5b7f-4faa-82c3-954a0b399be1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:49.416406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774ee7e2299b4a30ae78576e7298d87f', 'request_id': 'req-84115975-5b7f-4faa-82c3-954a0b399be1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:49.424473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774ee7e2299b4a30ae78576e7298d87f', 'request_id': 'req-84115975-5b7f-4faa-82c3-954a0b399be1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:49.435409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] The token cache is being invalidate because project 774ee7e2299b4a30ae78576e7298d87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:25:49.435698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:49.436387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-84115975-5b7f-4faa-82c3-954a0b399be1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:49.437904 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1168/2351] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:25:49 2022] DELETE /identity/v3/projects/774ee7e2299b4a30ae78576e7298d87f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:49.443256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff32569-0f5a-4ef6-86b8-b91b66d81b22 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:49.443540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff32569-0f5a-4ef6-86b8-b91b66d81b22 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:49.443847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff32569-0f5a-4ef6-86b8-b91b66d81b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:49.464026 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-1ff32569-0f5a-4ef6-86b8-b91b66d81b22 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:49.465844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1ff32569-0f5a-4ef6-86b8-b91b66d81b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:49.525719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1184/2352] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:49.533031 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a826bdb-5df0-48a8-aff8-7702560044af None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:49.533317 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a826bdb-5df0-48a8-aff8-7702560044af None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:49.533592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a826bdb-5df0-48a8-aff8-7702560044af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:49.556249 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7a826bdb-5df0-48a8-aff8-7702560044af None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:49.557503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7a826bdb-5df0-48a8-aff8-7702560044af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:49.565530 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1169/2353] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:25:49.572507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff32569-0f5a-4ef6-86b8-b91b66d81b22 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:49.582220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff32569-0f5a-4ef6-86b8-b91b66d81b22 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:49.586952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa34666-59e4-4081-854a-701d2603718f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:49.587462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa34666-59e4-4081-854a-701d2603718f admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:49.587802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa34666-59e4-4081-854a-701d2603718f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:49.588029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa34666-59e4-4081-854a-701d2603718f admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:49.588701 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa34666-59e4-4081-854a-701d2603718f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:25:49.589064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa34666-59e4-4081-854a-701d2603718f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:49.590725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa34666-59e4-4081-854a-701d2603718f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:49.607678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1185/2354] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:25:49.608676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a826bdb-5df0-48a8-aff8-7702560044af None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:49.618926 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a826bdb-5df0-48a8-aff8-7702560044af None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:49.623834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738eacb9-7227-4c60-8c84-88b38c010b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:49.624438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738eacb9-7227-4c60-8c84-88b38c010b42 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:49.624716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738eacb9-7227-4c60-8c84-88b38c010b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:49.624984 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738eacb9-7227-4c60-8c84-88b38c010b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:49.625629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738eacb9-7227-4c60-8c84-88b38c010b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:49.627511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738eacb9-7227-4c60-8c84-88b38c010b42 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:49.720410 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1170/2355] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:25:49 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:25:49.723567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa34666-59e4-4081-854a-701d2603718f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:49.729592 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:49.737767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa34666-59e4-4081-854a-701d2603718f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:49.810998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f695f12-f529-4fec-9c3c-21f5b955bc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:49.812370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f695f12-f529-4fec-9c3c-21f5b955bc42 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:49.812874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f695f12-f529-4fec-9c3c-21f5b955bc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:49.813037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f695f12-f529-4fec-9c3c-21f5b955bc42 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:49.813798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f695f12-f529-4fec-9c3c-21f5b955bc42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:49.816220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f695f12-f529-4fec-9c3c-21f5b955bc42 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:49.937356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1186/2356] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:25:49 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:25:49.942139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738eacb9-7227-4c60-8c84-88b38c010b42 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:49.960728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738eacb9-7227-4c60-8c84-88b38c010b42 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:49.965583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798c916-2a4b-42ed-87ae-ac5e5a5e4ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:49.966215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798c916-2a4b-42ed-87ae-ac5e5a5e4ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:49.966524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798c916-2a4b-42ed-87ae-ac5e5a5e4ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:49.966796 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798c916-2a4b-42ed-87ae-ac5e5a5e4ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:49.967461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798c916-2a4b-42ed-87ae-ac5e5a5e4ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:49.969109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798c916-2a4b-42ed-87ae-ac5e5a5e4ee0 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:49.980879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1171/2357] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:25:49.984930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f695f12-f529-4fec-9c3c-21f5b955bc42 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:50.000803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f695f12-f529-4fec-9c3c-21f5b955bc42 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:50.006460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d654844-980e-460c-9e86-658bb049586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:50.007585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d654844-980e-460c-9e86-658bb049586e admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:50.007911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d654844-980e-460c-9e86-658bb049586e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:50.008185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d654844-980e-460c-9e86-658bb049586e admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:50.009320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d654844-980e-460c-9e86-658bb049586e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:50.013516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d654844-980e-460c-9e86-658bb049586e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:50.027585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1187/2358] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:25:50.029822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798c916-2a4b-42ed-87ae-ac5e5a5e4ee0 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:50.040380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798c916-2a4b-42ed-87ae-ac5e5a5e4ee0 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:50.045301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1b14b-2069-48c1-ba07-0a9b26260e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:50.045937 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b14b-2069-48c1-ba07-0a9b26260e19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:50.046211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b14b-2069-48c1-ba07-0a9b26260e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:50.046480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b14b-2069-48c1-ba07-0a9b26260e19 admin admin] PATH_INFO: `/v3/projects/0ec269dd4dc448cbab79cf60daed31f4/users/4c136a7e50f845a9a2e30445c557a672/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:50.062189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1b14b-2069-48c1-ba07-0a9b26260e19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec269dd4dc448cbab79cf60daed31f4, user_id=4c136a7e50f845a9a2e30445c557a672, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:50.067802 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d1b14b-2069-48c1-ba07-0a9b26260e19 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:50.147298 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1172/2359] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:25:50 2022] PUT /identity/v3/projects/0ec269dd4dc448cbab79cf60daed31f4/users/4c136a7e50f845a9a2e30445c557a672/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:25:50.160313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069f08e-ec96-42fb-b5f5-6e1b64967612 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:50.161104 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069f08e-ec96-42fb-b5f5-6e1b64967612 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:50.161380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069f08e-ec96-42fb-b5f5-6e1b64967612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:50.193735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-c069f08e-ec96-42fb-b5f5-6e1b64967612 None None] MFA Rules not processed for user `4c136a7e50f845a9a2e30445c557a672`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:50.194971 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c069f08e-ec96-42fb-b5f5-6e1b64967612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:50.271389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1188/2360] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:50.274453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1b14b-2069-48c1-ba07-0a9b26260e19 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:50.276291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-50d1b14b-2069-48c1-ba07-0a9b26260e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:50.336481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d1b14b-2069-48c1-ba07-0a9b26260e19 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:50.340537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bacb3-b293-40bb-aeb8-bcc7f734e388 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:50.341686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bacb3-b293-40bb-aeb8-bcc7f734e388 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:50.341993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bacb3-b293-40bb-aeb8-bcc7f734e388 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:50.342287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bacb3-b293-40bb-aeb8-bcc7f734e388 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:50.352970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bacb3-b293-40bb-aeb8-bcc7f734e388 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:50.354440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bacb3-b293-40bb-aeb8-bcc7f734e388 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:50.363590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1173/2361] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:52.146611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069f08e-ec96-42fb-b5f5-6e1b64967612 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:52.148149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c069f08e-ec96-42fb-b5f5-6e1b64967612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:52.227120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069f08e-ec96-42fb-b5f5-6e1b64967612 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:52.232388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4eaaa2-0a58-4894-8cc7-88c63e7c26e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:52.233250 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eaaa2-0a58-4894-8cc7-88c63e7c26e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:52.233623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eaaa2-0a58-4894-8cc7-88c63e7c26e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:52.234115 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eaaa2-0a58-4894-8cc7-88c63e7c26e2 admin admin] PATH_INFO: `/v3/users/4bd826253c6b4b5096bee7ffaac04d18` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:52.235372 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eaaa2-0a58-4894-8cc7-88c63e7c26e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd826253c6b4b5096bee7ffaac04d18)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:52.238157 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eaaa2-0a58-4894-8cc7-88c63e7c26e2 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:52.254945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:52.469155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1a4eaaa2-0a58-4894-8cc7-88c63e7c26e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd826253c6b4b5096bee7ffaac04d18'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:52.552054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1a4eaaa2-0a58-4894-8cc7-88c63e7c26e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd826253c6b4b5096bee7ffaac04d18'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:52.552596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1a4eaaa2-0a58-4894-8cc7-88c63e7c26e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd826253c6b4b5096bee7ffaac04d18'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:52.553195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1a4eaaa2-0a58-4894-8cc7-88c63e7c26e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd826253c6b4b5096bee7ffaac04d18'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:52.560668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1a4eaaa2-0a58-4894-8cc7-88c63e7c26e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd826253c6b4b5096bee7ffaac04d18'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:52.578609 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1189/2362] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:25:52 2022] DELETE /identity/v3/users/4bd826253c6b4b5096bee7ffaac04d18 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:52.873584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bacb3-b293-40bb-aeb8-bcc7f734e388 service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:52.884496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bacb3-b293-40bb-aeb8-bcc7f734e388 service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:52.914088 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:52.955260 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:52.955943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:52.956229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:52.956510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] PATH_INFO: `/v3/projects/146ccff43bd84cf1bc84b9b862190678` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:52.957670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=146ccff43bd84cf1bc84b9b862190678)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:52.959844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:53.094509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '146ccff43bd84cf1bc84b9b862190678', 'request_id': 'req-7e3817dc-ff31-4d5e-a610-bb8896fed824'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:53.174822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '146ccff43bd84cf1bc84b9b862190678', 'request_id': 'req-7e3817dc-ff31-4d5e-a610-bb8896fed824'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:53.200483 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '146ccff43bd84cf1bc84b9b862190678', 'request_id': 'req-7e3817dc-ff31-4d5e-a610-bb8896fed824'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:53.205080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] The token cache is being invalidate because project 146ccff43bd84cf1bc84b9b862190678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:25:53.205080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:53.205080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7e3817dc-ff31-4d5e-a610-bb8896fed824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:53.207380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1174/2363] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:25:52 2022] DELETE /identity/v3/projects/146ccff43bd84cf1bc84b9b862190678 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:53.212316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10854721-851f-457a-8804-6cfcf91d3c02 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:53.212695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10854721-851f-457a-8804-6cfcf91d3c02 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:53.213047 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10854721-851f-457a-8804-6cfcf91d3c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:53.237243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-10854721-851f-457a-8804-6cfcf91d3c02 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:53.238608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-10854721-851f-457a-8804-6cfcf91d3c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:53.308126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1190/2364] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:53.319500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ef73b-15bd-4932-a5c9-895218134646 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:53.320142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ef73b-15bd-4932-a5c9-895218134646 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:53.320425 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ef73b-15bd-4932-a5c9-895218134646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:53.349161 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-613ef73b-15bd-4932-a5c9-895218134646 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:53.349161 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-613ef73b-15bd-4932-a5c9-895218134646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:53.373209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1175/2365] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:53.379964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10854721-851f-457a-8804-6cfcf91d3c02 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:53.391650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10854721-851f-457a-8804-6cfcf91d3c02 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:53.397022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c80c3-3de8-4e8c-b799-91a76879ee7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:53.397642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c80c3-3de8-4e8c-b799-91a76879ee7e admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:53.399986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c80c3-3de8-4e8c-b799-91a76879ee7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:53.399986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c80c3-3de8-4e8c-b799-91a76879ee7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:53.399986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c80c3-3de8-4e8c-b799-91a76879ee7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:25:53.399986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c80c3-3de8-4e8c-b799-91a76879ee7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:53.400906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c80c3-3de8-4e8c-b799-91a76879ee7e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:53.417194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1191/2366] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:25:53.421401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ef73b-15bd-4932-a5c9-895218134646 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:53.430884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ef73b-15bd-4932-a5c9-895218134646 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:53.436825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7942bed-ce8f-4f6b-8729-6e6b38081a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:53.437427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7942bed-ce8f-4f6b-8729-6e6b38081a01 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:53.437709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7942bed-ce8f-4f6b-8729-6e6b38081a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:53.437985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7942bed-ce8f-4f6b-8729-6e6b38081a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:53.438619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7942bed-ce8f-4f6b-8729-6e6b38081a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:53.440629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7942bed-ce8f-4f6b-8729-6e6b38081a01 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:53.502939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1176/2367] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:25:53 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:25:53.507221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c80c3-3de8-4e8c-b799-91a76879ee7e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:53.517367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c80c3-3de8-4e8c-b799-91a76879ee7e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:53.582161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c027a-ade1-4e84-8be7-06083a5cb7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:53.583538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c027a-ade1-4e84-8be7-06083a5cb7dd admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:53.583851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c027a-ade1-4e84-8be7-06083a5cb7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:53.584123 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c027a-ade1-4e84-8be7-06083a5cb7dd admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:53.584763 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c027a-ade1-4e84-8be7-06083a5cb7dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:53.586688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c027a-ade1-4e84-8be7-06083a5cb7dd admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:53.691447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1192/2368] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:25:53 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:25:53.696311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7942bed-ce8f-4f6b-8729-6e6b38081a01 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:53.708266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7942bed-ce8f-4f6b-8729-6e6b38081a01 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:53.713035 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bcccb-8144-4184-8628-6e4bac4c48d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:53.713795 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bcccb-8144-4184-8628-6e4bac4c48d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:53.714073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bcccb-8144-4184-8628-6e4bac4c48d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:53.714353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bcccb-8144-4184-8628-6e4bac4c48d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:53.714996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bcccb-8144-4184-8628-6e4bac4c48d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:53.716623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bcccb-8144-4184-8628-6e4bac4c48d7 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:53.730754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1177/2369] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:25:53.733033 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c027a-ade1-4e84-8be7-06083a5cb7dd admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:53.748800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c027a-ade1-4e84-8be7-06083a5cb7dd admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:53.753564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403d287-b722-457c-834e-388b5ae24898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:53.754179 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403d287-b722-457c-834e-388b5ae24898 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:53.754448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403d287-b722-457c-834e-388b5ae24898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:53.755259 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403d287-b722-457c-834e-388b5ae24898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:53.755977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403d287-b722-457c-834e-388b5ae24898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:53.757565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403d287-b722-457c-834e-388b5ae24898 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:53.759083 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:53.771966 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1193/2370] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:25:53.776411 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bcccb-8144-4184-8628-6e4bac4c48d7 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:53.789614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bcccb-8144-4184-8628-6e4bac4c48d7 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:53.795062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c9cd1-84aa-4b09-81fd-80849e2e8c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:53.795724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c9cd1-84aa-4b09-81fd-80849e2e8c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:53.796105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c9cd1-84aa-4b09-81fd-80849e2e8c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:53.796406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c9cd1-84aa-4b09-81fd-80849e2e8c27 admin admin] PATH_INFO: `/v3/projects/b53bc57c89fd4cad98ee6dae3a952ffa/users/bc60a1ee965c4d22a2ea93004dc97229/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:53.812800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c9cd1-84aa-4b09-81fd-80849e2e8c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b53bc57c89fd4cad98ee6dae3a952ffa, user_id=bc60a1ee965c4d22a2ea93004dc97229, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:53.817546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c9cd1-84aa-4b09-81fd-80849e2e8c27 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:53.914758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1178/2371] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:25:53 2022] PUT /identity/v3/projects/b53bc57c89fd4cad98ee6dae3a952ffa/users/bc60a1ee965c4d22a2ea93004dc97229/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:53.926048 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d8f66-45ad-4cd1-98b8-2ba4b4eb43db None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:53.926372 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d8f66-45ad-4cd1-98b8-2ba4b4eb43db None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:53.926611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d8f66-45ad-4cd1-98b8-2ba4b4eb43db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:53.963865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e00d8f66-45ad-4cd1-98b8-2ba4b4eb43db None None] MFA Rules not processed for user `bc60a1ee965c4d22a2ea93004dc97229`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:53.965247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e00d8f66-45ad-4cd1-98b8-2ba4b4eb43db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:54.043364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1194/2372] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:54.056884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c9cd1-84aa-4b09-81fd-80849e2e8c27 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:54.058982 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-683c9cd1-84aa-4b09-81fd-80849e2e8c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:54.118278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:54.122159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c9cd1-84aa-4b09-81fd-80849e2e8c27 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:54.126196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3555c-f6df-400d-9fb3-d181f52e7ce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:54.126815 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3555c-f6df-400d-9fb3-d181f52e7ce4 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:54.127351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3555c-f6df-400d-9fb3-d181f52e7ce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:54.127351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3555c-f6df-400d-9fb3-d181f52e7ce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:54.136926 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3555c-f6df-400d-9fb3-d181f52e7ce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:54.138435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3555c-f6df-400d-9fb3-d181f52e7ce4 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:54.148213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1179/2373] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:55.549080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d8f66-45ad-4cd1-98b8-2ba4b4eb43db None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:55.550497 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e00d8f66-45ad-4cd1-98b8-2ba4b4eb43db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:55.620147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d8f66-45ad-4cd1-98b8-2ba4b4eb43db None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:55.624747 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:55.625391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:55.625673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:55.625978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] PATH_INFO: `/v3/users/4c136a7e50f845a9a2e30445c557a672` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:55.627100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c136a7e50f845a9a2e30445c557a672)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:55.629062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:55.887869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c136a7e50f845a9a2e30445c557a672'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:55.974601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c136a7e50f845a9a2e30445c557a672'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:55.975278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c136a7e50f845a9a2e30445c557a672'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:55.975977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c136a7e50f845a9a2e30445c557a672'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:55.983822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c136a7e50f845a9a2e30445c557a672'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:56.001260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1195/2374] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:25:55 2022] DELETE /identity/v3/users/4c136a7e50f845a9a2e30445c557a672 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:56.009482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6aaee-40b9-4364-abb3-8187dad5af6d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:56.010017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6aaee-40b9-4364-abb3-8187dad5af6d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:56.010017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6aaee-40b9-4364-abb3-8187dad5af6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:56.030607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-d2d6aaee-40b9-4364-abb3-8187dad5af6d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:56.031847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d2d6aaee-40b9-4364-abb3-8187dad5af6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:56.090508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1180/2375] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:56.101816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:56.103247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:56.109804 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:56.160384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f58dc7-251c-4da9-a874-c34fbdfd2241 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:56.164215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eee728-3be7-41e1-844e-0b9fe20b242b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:56.164837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eee728-3be7-41e1-844e-0b9fe20b242b service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:56.165115 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eee728-3be7-41e1-844e-0b9fe20b242b service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:56.165387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eee728-3be7-41e1-844e-0b9fe20b242b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:56.175448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eee728-3be7-41e1-844e-0b9fe20b242b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:56.177136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eee728-3be7-41e1-844e-0b9fe20b242b service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:56.190063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1196/2376] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:25:56.663093 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6aaee-40b9-4364-abb3-8187dad5af6d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:56.665351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d2d6aaee-40b9-4364-abb3-8187dad5af6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:56.680244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6aaee-40b9-4364-abb3-8187dad5af6d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:56.686793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:56.686793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:56.686793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:56.686793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] PATH_INFO: `/v3/projects/0ec269dd4dc448cbab79cf60daed31f4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:56.688029 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec269dd4dc448cbab79cf60daed31f4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:56.690020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:56.856625 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec269dd4dc448cbab79cf60daed31f4', 'request_id': 'req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:56.928791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec269dd4dc448cbab79cf60daed31f4', 'request_id': 'req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:56.936124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec269dd4dc448cbab79cf60daed31f4', 'request_id': 'req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:56.948332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] The token cache is being invalidate because project 0ec269dd4dc448cbab79cf60daed31f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:25:56.948663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:56.949303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8024cd79-2472-41b6-a47b-3f8ba2d24bd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:56.951053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1181/2377] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:25:56 2022] DELETE /identity/v3/projects/0ec269dd4dc448cbab79cf60daed31f4 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:25:58.307813 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eee728-3be7-41e1-844e-0b9fe20b242b service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:58.309227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-31eee728-3be7-41e1-844e-0b9fe20b242b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:58.321898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eee728-3be7-41e1-844e-0b9fe20b242b service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:58.380447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:58.381100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:58.381380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:58.381657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] PATH_INFO: `/v3/users/bc60a1ee965c4d22a2ea93004dc97229` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:58.382778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc60a1ee965c4d22a2ea93004dc97229)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:58.384746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:58.571687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc60a1ee965c4d22a2ea93004dc97229'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:58.643302 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc60a1ee965c4d22a2ea93004dc97229'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:58.643958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc60a1ee965c4d22a2ea93004dc97229'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:58.644555 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc60a1ee965c4d22a2ea93004dc97229'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:58.652496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc60a1ee965c4d22a2ea93004dc97229'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:58.672527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1197/2378] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:25:58 2022] DELETE /identity/v3/users/bc60a1ee965c4d22a2ea93004dc97229 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:58.677184 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846018a8-fcca-465b-bd35-cfcc3a49f03d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:58.677670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846018a8-fcca-465b-bd35-cfcc3a49f03d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:58.677763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846018a8-fcca-465b-bd35-cfcc3a49f03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:58.696684 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-846018a8-fcca-465b-bd35-cfcc3a49f03d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:58.697908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-846018a8-fcca-465b-bd35-cfcc3a49f03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:58.759428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1182/2379] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:25:58.775416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:58.777768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:58.812952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:58.842382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd61c2d-9f26-4343-8cb4-88687694ceeb admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:58.847069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5743e-3e55-4286-932e-c510a51875bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:58.847694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5743e-3e55-4286-932e-c510a51875bb service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:58.849151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5743e-3e55-4286-932e-c510a51875bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:58.849151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5743e-3e55-4286-932e-c510a51875bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:58.865176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5743e-3e55-4286-932e-c510a51875bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:58.866629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5743e-3e55-4286-932e-c510a51875bb service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:58.876529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1198/2380] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:25:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:25:59.332105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846018a8-fcca-465b-bd35-cfcc3a49f03d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:59.333580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-846018a8-fcca-465b-bd35-cfcc3a49f03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:59.348128 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846018a8-fcca-465b-bd35-cfcc3a49f03d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:59.353510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:59.355499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:59.355808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:59.356183 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] PATH_INFO: `/v3/projects/b53bc57c89fd4cad98ee6dae3a952ffa` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:59.357315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b53bc57c89fd4cad98ee6dae3a952ffa)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:59.359536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:59.381228 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:25:59.493479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b53bc57c89fd4cad98ee6dae3a952ffa', 'request_id': 'req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:59.582175 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b53bc57c89fd4cad98ee6dae3a952ffa', 'request_id': 'req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:59.588569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b53bc57c89fd4cad98ee6dae3a952ffa', 'request_id': 'req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:59.598849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] The token cache is being invalidate because project b53bc57c89fd4cad98ee6dae3a952ffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:25:59.599142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:59.599804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-7552a6d3-34e1-4dd2-bf45-3ff18d9bad4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:25:59.601326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1183/2381] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:25:59 2022] DELETE /identity/v3/projects/b53bc57c89fd4cad98ee6dae3a952ffa => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:25:59.606249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4a0e3-788e-4018-aef3-a1a8a9e7db53 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:59.606697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4a0e3-788e-4018-aef3-a1a8a9e7db53 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:59.606789 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4a0e3-788e-4018-aef3-a1a8a9e7db53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:59.626699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-d0d4a0e3-788e-4018-aef3-a1a8a9e7db53 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:59.627945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d0d4a0e3-788e-4018-aef3-a1a8a9e7db53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:59.686957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1199/2382] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:59.695926 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df292a96-789a-4eb0-a3c7-d28db3211d18 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:59.696388 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df292a96-789a-4eb0-a3c7-d28db3211d18 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:59.696512 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df292a96-789a-4eb0-a3c7-d28db3211d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:59.716264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-df292a96-789a-4eb0-a3c7-d28db3211d18 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:25:59.717474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-df292a96-789a-4eb0-a3c7-d28db3211d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:25:59.725751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1184/2383] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:25:59.729897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4a0e3-788e-4018-aef3-a1a8a9e7db53 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:59.740224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d4a0e3-788e-4018-aef3-a1a8a9e7db53 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:59.744777 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa7cb2-f781-441f-9002-e69f4da0dbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:59.745401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa7cb2-f781-441f-9002-e69f4da0dbe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:59.745681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa7cb2-f781-441f-9002-e69f4da0dbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:59.745958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa7cb2-f781-441f-9002-e69f4da0dbe2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:59.746602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa7cb2-f781-441f-9002-e69f4da0dbe2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:25:59.746904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa7cb2-f781-441f-9002-e69f4da0dbe2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:59.748542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa7cb2-f781-441f-9002-e69f4da0dbe2 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:59.760183 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1200/2384] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:25:59.764667 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df292a96-789a-4eb0-a3c7-d28db3211d18 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:59.773588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df292a96-789a-4eb0-a3c7-d28db3211d18 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:59.778267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e52b2-a732-46ee-a52a-9cf5986cbf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:59.778850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e52b2-a732-46ee-a52a-9cf5986cbf64 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:59.779126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e52b2-a732-46ee-a52a-9cf5986cbf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:59.779396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e52b2-a732-46ee-a52a-9cf5986cbf64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:59.780049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e52b2-a732-46ee-a52a-9cf5986cbf64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:59.782041 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e52b2-a732-46ee-a52a-9cf5986cbf64 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:25:59.851660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1185/2385] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:25:59 2022] POST /identity/v3/projects => generated 375 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:25:59.856345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa7cb2-f781-441f-9002-e69f4da0dbe2 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:25:59.867521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa7cb2-f781-441f-9002-e69f4da0dbe2 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:25:59.931852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c874793-bd89-40c3-b0a9-0a4c46a0919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:25:59.932459 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c874793-bd89-40c3-b0a9-0a4c46a0919e admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:25:59.932734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c874793-bd89-40c3-b0a9-0a4c46a0919e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:25:59.933005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c874793-bd89-40c3-b0a9-0a4c46a0919e admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:25:59.933624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c874793-bd89-40c3-b0a9-0a4c46a0919e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:25:59.935557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c874793-bd89-40c3-b0a9-0a4c46a0919e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:00.043982 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1201/2386] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:25:59 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:26:00.045331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e52b2-a732-46ee-a52a-9cf5986cbf64 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:00.055920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e52b2-a732-46ee-a52a-9cf5986cbf64 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:00.060556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f0e73-d36b-421c-b675-5b90843f935e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:00.061678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f0e73-d36b-421c-b675-5b90843f935e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:00.061678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f0e73-d36b-421c-b675-5b90843f935e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:00.061678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f0e73-d36b-421c-b675-5b90843f935e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:00.062461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f0e73-d36b-421c-b675-5b90843f935e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:00.064094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f0e73-d36b-421c-b675-5b90843f935e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:00.075047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1186/2387] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:26:00.086560 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c874793-bd89-40c3-b0a9-0a4c46a0919e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:00.115305 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c874793-bd89-40c3-b0a9-0a4c46a0919e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:00.116027 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28064cc8-5730-4c0b-a830-9cdd72e8fbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:00.116643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28064cc8-5730-4c0b-a830-9cdd72e8fbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:00.117413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28064cc8-5730-4c0b-a830-9cdd72e8fbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:00.117714 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28064cc8-5730-4c0b-a830-9cdd72e8fbe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:00.118390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28064cc8-5730-4c0b-a830-9cdd72e8fbe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:00.120054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28064cc8-5730-4c0b-a830-9cdd72e8fbe0 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:00.131210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1202/2388] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:26:00.136082 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f0e73-d36b-421c-b675-5b90843f935e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:00.148506 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f0e73-d36b-421c-b675-5b90843f935e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:00.153242 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15eee2c-3003-4499-bf79-d1b365060f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:00.153924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eee2c-3003-4499-bf79-d1b365060f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:00.154205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eee2c-3003-4499-bf79-d1b365060f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:00.154485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15eee2c-3003-4499-bf79-d1b365060f53 admin admin] PATH_INFO: `/v3/projects/5fd13798945b4540a1e368011646ee95/users/ff1021e03c9b41bdbc9f3e224201091e/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:00.171861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15eee2c-3003-4499-bf79-d1b365060f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd13798945b4540a1e368011646ee95, user_id=ff1021e03c9b41bdbc9f3e224201091e, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:00.174531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15eee2c-3003-4499-bf79-d1b365060f53 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:00.278055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1187/2389] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:26:00 2022] PUT /identity/v3/projects/5fd13798945b4540a1e368011646ee95/users/ff1021e03c9b41bdbc9f3e224201091e/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:26:00.289303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5ead-efd5-4b9d-aec2-ae08c3d12a51 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:00.289596 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5ead-efd5-4b9d-aec2-ae08c3d12a51 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:00.289871 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a5ead-efd5-4b9d-aec2-ae08c3d12a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:00.320230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-2f1a5ead-efd5-4b9d-aec2-ae08c3d12a51 None None] MFA Rules not processed for user `ff1021e03c9b41bdbc9f3e224201091e`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:00.321550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2f1a5ead-efd5-4b9d-aec2-ae08c3d12a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:00.347489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:26:00.397619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1203/2390] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:26:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:00.408723 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15eee2c-3003-4499-bf79-d1b365060f53 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:00.410401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f15eee2c-3003-4499-bf79-d1b365060f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:00.480129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15eee2c-3003-4499-bf79-d1b365060f53 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:00.483955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05392c44-6df3-4c5a-8263-3427e3d223d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:00.484569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05392c44-6df3-4c5a-8263-3427e3d223d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:00.484846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05392c44-6df3-4c5a-8263-3427e3d223d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:00.485122 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05392c44-6df3-4c5a-8263-3427e3d223d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:00.495146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05392c44-6df3-4c5a-8263-3427e3d223d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:00.497285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05392c44-6df3-4c5a-8263-3427e3d223d1 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:00.509970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1188/2391] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:26:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:26:02.970394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d036a7-6acf-4d8a-ad7f-2717a09c0567 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:02.970892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d036a7-6acf-4d8a-ad7f-2717a09c0567 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:02.971016 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d036a7-6acf-4d8a-ad7f-2717a09c0567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:02.990973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-03d036a7-6acf-4d8a-ad7f-2717a09c0567 None None] MFA Rules not processed for user `d3da95b627b74c28b921586d0da89521`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:02.992195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-03d036a7-6acf-4d8a-ad7f-2717a09c0567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:03.054126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1204/2392] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:26:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:10.504613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:26:13.047255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=41 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:26:14.850225 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b3bd3-dd0a-4734-86ec-ca626b2fabdb None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:14.850792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b3bd3-dd0a-4734-86ec-ca626b2fabdb None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:14.850892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b3bd3-dd0a-4734-86ec-ca626b2fabdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:14.869796 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-473b3bd3-dd0a-4734-86ec-ca626b2fabdb None None] MFA Rules not processed for user `be10df0a4d1d42208c46615d6ddd970c`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:14.870999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-473b3bd3-dd0a-4734-86ec-ca626b2fabdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:14.925716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1189/2393] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:26:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:15.518519 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d036a7-6acf-4d8a-ad7f-2717a09c0567 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:15.587622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d036a7-6acf-4d8a-ad7f-2717a09c0567 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:15.592495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:15.594762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:15.594762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:15.594762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] PATH_INFO: `/v3/users/ff1021e03c9b41bdbc9f3e224201091e` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:15.594762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff1021e03c9b41bdbc9f3e224201091e)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:15.596934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:15.739645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff1021e03c9b41bdbc9f3e224201091e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:26:15.812117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff1021e03c9b41bdbc9f3e224201091e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:26:15.812752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff1021e03c9b41bdbc9f3e224201091e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:26:15.813371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff1021e03c9b41bdbc9f3e224201091e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:26:15.821641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff1021e03c9b41bdbc9f3e224201091e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:26:15.863509 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1205/2394] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:26:15 2022] DELETE /identity/v3/users/ff1021e03c9b41bdbc9f3e224201091e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:26:15.868678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f42a7-653d-4b93-9062-9fc45d88a46b None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:15.868678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f42a7-653d-4b93-9062-9fc45d88a46b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:15.869328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f42a7-653d-4b93-9062-9fc45d88a46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:15.892982 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-006f42a7-653d-4b93-9062-9fc45d88a46b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:15.894300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-006f42a7-653d-4b93-9062-9fc45d88a46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:15.965914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1190/2395] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:15.976331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:15.977776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:16.036270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85537fd9-5a4d-4bf4-8684-1bc1e467b69c admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:16.039945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c43ff6-3927-41c3-a144-0cbda077cf93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:16.040549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c43ff6-3927-41c3-a144-0cbda077cf93 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:16.040828 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c43ff6-3927-41c3-a144-0cbda077cf93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:16.041101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c43ff6-3927-41c3-a144-0cbda077cf93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:16.050370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c43ff6-3927-41c3-a144-0cbda077cf93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:16.051825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c43ff6-3927-41c3-a144-0cbda077cf93 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:16.061603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1206/2396] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:26:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:26:16.488860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f42a7-653d-4b93-9062-9fc45d88a46b None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:16.490681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-006f42a7-653d-4b93-9062-9fc45d88a46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:16.514549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f42a7-653d-4b93-9062-9fc45d88a46b None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:16.543666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:16.545049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:16.545875 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:16.546295 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] PATH_INFO: `/v3/projects/5fd13798945b4540a1e368011646ee95` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:16.548364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd13798945b4540a1e368011646ee95)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:16.550696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:16.672049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd13798945b4540a1e368011646ee95', 'request_id': 'req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:26:16.772952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd13798945b4540a1e368011646ee95', 'request_id': 'req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:26:16.780265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd13798945b4540a1e368011646ee95', 'request_id': 'req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:26:16.795112 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] The token cache is being invalidate because project 5fd13798945b4540a1e368011646ee95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:26:16.795666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:26:16.796367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-aaa267ed-31fc-4bcd-ab46-1eabd5cd74be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:26:16.797850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1191/2397] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:26:16 2022] DELETE /identity/v3/projects/5fd13798945b4540a1e368011646ee95 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:26:16.804538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc07300-c723-4fe4-bc96-2380737fa96b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:16.805067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc07300-c723-4fe4-bc96-2380737fa96b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:16.805067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc07300-c723-4fe4-bc96-2380737fa96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:16.824788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-3dc07300-c723-4fe4-bc96-2380737fa96b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:16.825985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-3dc07300-c723-4fe4-bc96-2380737fa96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:16.845443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:26:16.899549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1207/2398] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:26:16.908019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d4160-e95a-469b-8c2c-f82e6978744f None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:16.908537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d4160-e95a-469b-8c2c-f82e6978744f None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:16.908650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d4160-e95a-469b-8c2c-f82e6978744f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:16.929174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-451d4160-e95a-469b-8c2c-f82e6978744f None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:16.930307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-451d4160-e95a-469b-8c2c-f82e6978744f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:16.942818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1192/2399] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:16.957600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc07300-c723-4fe4-bc96-2380737fa96b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:16.970434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc07300-c723-4fe4-bc96-2380737fa96b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:16.975726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681e399-4296-4cd9-a49a-8ae0254befea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:16.976400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681e399-4296-4cd9-a49a-8ae0254befea admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:16.976672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681e399-4296-4cd9-a49a-8ae0254befea admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:16.976942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681e399-4296-4cd9-a49a-8ae0254befea admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:16.977591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681e399-4296-4cd9-a49a-8ae0254befea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:26:16.977897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681e399-4296-4cd9-a49a-8ae0254befea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:16.979499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681e399-4296-4cd9-a49a-8ae0254befea admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:16.991814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1208/2400] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:26:16.998819 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d4160-e95a-469b-8c2c-f82e6978744f None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:17.007149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d4160-e95a-469b-8c2c-f82e6978744f None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:17.013843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a861a28-789e-4fd7-b379-05487e6a1b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:17.014452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a861a28-789e-4fd7-b379-05487e6a1b53 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:17.015120 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a861a28-789e-4fd7-b379-05487e6a1b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:17.015229 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a861a28-789e-4fd7-b379-05487e6a1b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:17.015972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a861a28-789e-4fd7-b379-05487e6a1b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:17.018017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a861a28-789e-4fd7-b379-05487e6a1b53 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:17.082978 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:26:17.090850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1193/2401] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:26:16 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:26:17.094695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681e399-4296-4cd9-a49a-8ae0254befea admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:17.105702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681e399-4296-4cd9-a49a-8ae0254befea admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:17.162466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447007b6-50b7-44e7-98c0-99ec4ea507ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:17.163086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447007b6-50b7-44e7-98c0-99ec4ea507ad admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:17.163373 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447007b6-50b7-44e7-98c0-99ec4ea507ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:17.163647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447007b6-50b7-44e7-98c0-99ec4ea507ad admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:17.164314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447007b6-50b7-44e7-98c0-99ec4ea507ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:17.166274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447007b6-50b7-44e7-98c0-99ec4ea507ad admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:17.250994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1209/2402] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:26:17 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:26:17.254807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a861a28-789e-4fd7-b379-05487e6a1b53 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:17.266193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a861a28-789e-4fd7-b379-05487e6a1b53 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:17.271018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069ec0c-6fde-4191-bc00-adaf74071ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:17.271620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069ec0c-6fde-4191-bc00-adaf74071ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:17.272508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069ec0c-6fde-4191-bc00-adaf74071ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:17.272788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069ec0c-6fde-4191-bc00-adaf74071ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:17.273463 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069ec0c-6fde-4191-bc00-adaf74071ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:17.275063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e069ec0c-6fde-4191-bc00-adaf74071ac6 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:17.288861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1194/2403] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:26:17.292765 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447007b6-50b7-44e7-98c0-99ec4ea507ad admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:17.304086 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447007b6-50b7-44e7-98c0-99ec4ea507ad admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:17.309737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355768a7-f250-47f4-9751-853b8368ff5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:17.310345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355768a7-f250-47f4-9751-853b8368ff5a admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:17.310626 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355768a7-f250-47f4-9751-853b8368ff5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:17.310901 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355768a7-f250-47f4-9751-853b8368ff5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:17.311548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355768a7-f250-47f4-9751-853b8368ff5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:17.313190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355768a7-f250-47f4-9751-853b8368ff5a admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:17.325951 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1210/2404] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:26:17.333192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069ec0c-6fde-4191-bc00-adaf74071ac6 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:17.343255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069ec0c-6fde-4191-bc00-adaf74071ac6 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:17.348354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43551b0-bfb6-47ca-8f41-d331d77803cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:17.349040 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43551b0-bfb6-47ca-8f41-d331d77803cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:17.349324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43551b0-bfb6-47ca-8f41-d331d77803cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:17.349605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43551b0-bfb6-47ca-8f41-d331d77803cf admin admin] PATH_INFO: `/v3/projects/3b54810a2eb44e48a9ae72d8b5e1b121/users/22402b1d1d9d489888883e1d2c7c60b5/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:17.364050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43551b0-bfb6-47ca-8f41-d331d77803cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b54810a2eb44e48a9ae72d8b5e1b121, user_id=22402b1d1d9d489888883e1d2c7c60b5, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:17.367326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43551b0-bfb6-47ca-8f41-d331d77803cf admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:17.423818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1195/2405] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:26:17 2022] PUT /identity/v3/projects/3b54810a2eb44e48a9ae72d8b5e1b121/users/22402b1d1d9d489888883e1d2c7c60b5/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:26:17.429429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e9979-227b-430b-a5a1-76e8c58c5fc4 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:17.429713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e9979-227b-430b-a5a1-76e8c58c5fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:17.429989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e9979-227b-430b-a5a1-76e8c58c5fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:17.449646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ee8e9979-227b-430b-a5a1-76e8c58c5fc4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:17.450793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ee8e9979-227b-430b-a5a1-76e8c58c5fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:17.519814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1211/2406] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:17.530538 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43551b0-bfb6-47ca-8f41-d331d77803cf admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:17.532738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c43551b0-bfb6-47ca-8f41-d331d77803cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:17.595816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43551b0-bfb6-47ca-8f41-d331d77803cf admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:17.599451 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79713ea-9261-42d4-8dfb-49e40501e2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:17.600113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79713ea-9261-42d4-8dfb-49e40501e2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:17.600382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79713ea-9261-42d4-8dfb-49e40501e2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:17.600651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79713ea-9261-42d4-8dfb-49e40501e2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:17.610078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79713ea-9261-42d4-8dfb-49e40501e2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:17.611684 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79713ea-9261-42d4-8dfb-49e40501e2f2 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:17.621804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1196/2407] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:26:24.113277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397d39c-4625-46b0-9228-a6f9387f7134 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:24.113742 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397d39c-4625-46b0-9228-a6f9387f7134 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:24.113891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397d39c-4625-46b0-9228-a6f9387f7134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:24.161459 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-f397d39c-4625-46b0-9228-a6f9387f7134 None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:24.162958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f397d39c-4625-46b0-9228-a6f9387f7134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:24.199090 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:26:24.261216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1212/2408] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:26:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:24.298052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79713ea-9261-42d4-8dfb-49e40501e2f2 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:24.300955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b79713ea-9261-42d4-8dfb-49e40501e2f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:24.387791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79713ea-9261-42d4-8dfb-49e40501e2f2 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:24.393362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381040b2-cc7f-445b-8ed8-5ee7bdd9bd48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:24.394137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381040b2-cc7f-445b-8ed8-5ee7bdd9bd48 service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:24.394516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381040b2-cc7f-445b-8ed8-5ee7bdd9bd48 service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:24.394891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381040b2-cc7f-445b-8ed8-5ee7bdd9bd48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:24.404523 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381040b2-cc7f-445b-8ed8-5ee7bdd9bd48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:24.406253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381040b2-cc7f-445b-8ed8-5ee7bdd9bd48 service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:24.417585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1197/2409] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:26:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:26:26.611695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9130097-e1e2-4e4c-b233-a618165daf16 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:26.614658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9130097-e1e2-4e4c-b233-a618165daf16 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:26.614658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9130097-e1e2-4e4c-b233-a618165daf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:26.670303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e9130097-e1e2-4e4c-b233-a618165daf16 None None] MFA Rules not processed for user `22402b1d1d9d489888883e1d2c7c60b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:26.672365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e9130097-e1e2-4e4c-b233-a618165daf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:26.812514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1213/2410] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:26:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:26.816597 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381040b2-cc7f-445b-8ed8-5ee7bdd9bd48 service swift] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:26.836307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381040b2-cc7f-445b-8ed8-5ee7bdd9bd48 service swift] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:26.845767 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e24a4-1852-488c-af67-bcd21389a59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:26.847078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e24a4-1852-488c-af67-bcd21389a59d admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:26.847580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e24a4-1852-488c-af67-bcd21389a59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:26.848102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e24a4-1852-488c-af67-bcd21389a59d admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:26.849152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e24a4-1852-488c-af67-bcd21389a59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:26.852663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e24a4-1852-488c-af67-bcd21389a59d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:26.959895 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1198/2411] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:26:26 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:26:26.960111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9130097-e1e2-4e4c-b233-a618165daf16 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:26.972494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9130097-e1e2-4e4c-b233-a618165daf16 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:27.081302 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e61412-4e92-4652-991e-ca905141a18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:27.082542 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e61412-4e92-4652-991e-ca905141a18e admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:27.082983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e61412-4e92-4652-991e-ca905141a18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:27.083414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e61412-4e92-4652-991e-ca905141a18e admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:27.084420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e61412-4e92-4652-991e-ca905141a18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:27.087833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e61412-4e92-4652-991e-ca905141a18e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:27.197354 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1214/2412] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:26:26 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:26:27.201750 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e24a4-1852-488c-af67-bcd21389a59d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:27.214728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e24a4-1852-488c-af67-bcd21389a59d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:27.228025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58f457-d501-4c9f-b6ad-2f47232163a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:27.230020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58f457-d501-4c9f-b6ad-2f47232163a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:27.230020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58f457-d501-4c9f-b6ad-2f47232163a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:27.230020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58f457-d501-4c9f-b6ad-2f47232163a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:27.230020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58f457-d501-4c9f-b6ad-2f47232163a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:27.231494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58f457-d501-4c9f-b6ad-2f47232163a2 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:27.255929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1199/2413] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:26:27.259498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e61412-4e92-4652-991e-ca905141a18e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:27.274810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e61412-4e92-4652-991e-ca905141a18e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:27.280438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7c9a1-8b9a-4897-83a4-13bf15c04d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:27.282415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7c9a1-8b9a-4897-83a4-13bf15c04d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:27.282827 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7c9a1-8b9a-4897-83a4-13bf15c04d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:27.283203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7c9a1-8b9a-4897-83a4-13bf15c04d8c admin admin] PATH_INFO: `/v3/projects/9b236904f41e453583d1c07ac3391af1/users/93f32e35eb0149a1a7aa3ab630022376/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:27.307354 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7c9a1-8b9a-4897-83a4-13bf15c04d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b236904f41e453583d1c07ac3391af1, user_id=93f32e35eb0149a1a7aa3ab630022376, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:27.310504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7c9a1-8b9a-4897-83a4-13bf15c04d8c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:27.403783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1215/2414] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:26:27 2022] PUT /identity/v3/projects/9b236904f41e453583d1c07ac3391af1/users/93f32e35eb0149a1a7aa3ab630022376/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:26:27.408953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58f457-d501-4c9f-b6ad-2f47232163a2 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:27.432447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58f457-d501-4c9f-b6ad-2f47232163a2 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:27.490800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:26:27.554468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c4eca-e8c2-4e16-9468-2767a2640948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:27.555333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c4eca-e8c2-4e16-9468-2767a2640948 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:27.555669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c4eca-e8c2-4e16-9468-2767a2640948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:27.555971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c4eca-e8c2-4e16-9468-2767a2640948 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:27.556615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c4eca-e8c2-4e16-9468-2767a2640948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:27.558198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c4eca-e8c2-4e16-9468-2767a2640948 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:27.580322 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7c9a1-8b9a-4897-83a4-13bf15c04d8c admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:27.580541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1200/2415] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:26:27.595483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7c9a1-8b9a-4897-83a4-13bf15c04d8c admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:27.603071 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbbcfa-902d-4b71-9e95-21e705e367aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:27.604015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbbcfa-902d-4b71-9e95-21e705e367aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:27.604365 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbbcfa-902d-4b71-9e95-21e705e367aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:27.604710 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbbcfa-902d-4b71-9e95-21e705e367aa admin admin] PATH_INFO: `/v3/projects/9b236904f41e453583d1c07ac3391af1/users/93f32e35eb0149a1a7aa3ab630022376/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:27.620911 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbbcfa-902d-4b71-9e95-21e705e367aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b236904f41e453583d1c07ac3391af1, user_id=93f32e35eb0149a1a7aa3ab630022376, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:27.630022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbbcfa-902d-4b71-9e95-21e705e367aa admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:27.673078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1216/2416] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:26:27 2022] PUT /identity/v3/projects/9b236904f41e453583d1c07ac3391af1/users/93f32e35eb0149a1a7aa3ab630022376/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:26:36.713422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de072d81-a634-4ab5-ae40-4cab62e51b69 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:36.714964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de072d81-a634-4ab5-ae40-4cab62e51b69 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:36.714964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de072d81-a634-4ab5-ae40-4cab62e51b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:36.755372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-de072d81-a634-4ab5-ae40-4cab62e51b69 None None] MFA Rules not processed for user `93f32e35eb0149a1a7aa3ab630022376`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:36.760865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-de072d81-a634-4ab5-ae40-4cab62e51b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:36.871370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1201/2417] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:26:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:36.888296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbbcfa-902d-4b71-9e95-21e705e367aa admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:36.890937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-57dbbcfa-902d-4b71-9e95-21e705e367aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:36.927128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:26:37.049129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbbcfa-902d-4b71-9e95-21e705e367aa admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:37.055707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52960f-e9ae-490f-a0ff-fd70c7360cec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:37.056418 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52960f-e9ae-490f-a0ff-fd70c7360cec service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:37.056700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52960f-e9ae-490f-a0ff-fd70c7360cec service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:37.056978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52960f-e9ae-490f-a0ff-fd70c7360cec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:37.216386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52960f-e9ae-490f-a0ff-fd70c7360cec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:37.218147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52960f-e9ae-490f-a0ff-fd70c7360cec service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:37.270292 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1217/2418] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:26:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:26:40.793643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52fa36-d70b-4083-ae6e-4199839b1ed5 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:40.794158 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52fa36-d70b-4083-ae6e-4199839b1ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:40.794158 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52fa36-d70b-4083-ae6e-4199839b1ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:40.821354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-eb52fa36-d70b-4083-ae6e-4199839b1ed5 None None] MFA Rules not processed for user `22402b1d1d9d489888883e1d2c7c60b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:40.822837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-eb52fa36-d70b-4083-ae6e-4199839b1ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:40.830328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1202/2419] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:26:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:26:41.085762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52960f-e9ae-490f-a0ff-fd70c7360cec service nova] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:41.148265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52960f-e9ae-490f-a0ff-fd70c7360cec service nova] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:41.152076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c0ae4-5796-406c-b5f9-e85c520e7fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:41.152690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c0ae4-5796-406c-b5f9-e85c520e7fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:41.152967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c0ae4-5796-406c-b5f9-e85c520e7fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:41.153238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c0ae4-5796-406c-b5f9-e85c520e7fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:41.155019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-254c0ae4-5796-406c-b5f9-e85c520e7fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:41.218084 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c0ae4-5796-406c-b5f9-e85c520e7fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:41.219553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c0ae4-5796-406c-b5f9-e85c520e7fe8 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:41.230910 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1218/2420] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:26:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:26:46.639538 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491ad5d-11c7-4a8d-b5a9-c5066bcce60b None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:46.640148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491ad5d-11c7-4a8d-b5a9-c5066bcce60b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:46.640148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491ad5d-11c7-4a8d-b5a9-c5066bcce60b None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:46.641718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1203/2421] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:26:46.648514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f75f-2e3b-4659-9de7-5287cfa31f4d None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:46.648961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f75f-2e3b-4659-9de7-5287cfa31f4d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:46.649055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31f75f-2e3b-4659-9de7-5287cfa31f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:46.660137 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:26:46.674281 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ce31f75f-2e3b-4659-9de7-5287cfa31f4d None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:46.676092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ce31f75f-2e3b-4659-9de7-5287cfa31f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:46.691898 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1219/2422] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:26:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:46.697368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491ad5d-11c7-4a8d-b5a9-c5066bcce60b None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:46.699096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-5491ad5d-11c7-4a8d-b5a9-c5066bcce60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:46.897007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491ad5d-11c7-4a8d-b5a9-c5066bcce60b None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:46.901149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3347d-103f-4d48-9b96-1dfec538f8c3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5d7cf7534941cb857a4ab5867d978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:46.901758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3347d-103f-4d48-9b96-1dfec538f8c3 service glance] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:46.902039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3347d-103f-4d48-9b96-1dfec538f8c3 service glance] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:46.902311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3347d-103f-4d48-9b96-1dfec538f8c3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:46.912287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3347d-103f-4d48-9b96-1dfec538f8c3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:46.915591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3347d-103f-4d48-9b96-1dfec538f8c3 service glance] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:46.929988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1204/2423] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:26:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:26:47.140320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e6843-0e1e-45bb-9b9a-4d1f7d395b8c None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:47.140620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e6843-0e1e-45bb-9b9a-4d1f7d395b8c None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:47.141045 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e6843-0e1e-45bb-9b9a-4d1f7d395b8c None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:47.142394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1220/2424] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:26:47.147067 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec6c9-5e19-4d2b-833a-8f80ab7618ae None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:47.147350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec6c9-5e19-4d2b-833a-8f80ab7618ae None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:47.147624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ec6c9-5e19-4d2b-833a-8f80ab7618ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:47.169849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-de7ec6c9-5e19-4d2b-833a-8f80ab7618ae None None] MFA Rules not processed for user `b6a6b52462624e7f84f479144353b51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:47.171890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-de7ec6c9-5e19-4d2b-833a-8f80ab7618ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:47.180027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1205/2425] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:26:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:26:47.192521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e6843-0e1e-45bb-9b9a-4d1f7d395b8c None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:47.204276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e6843-0e1e-45bb-9b9a-4d1f7d395b8c None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:47.209513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e9752-a6fa-4e69-b06c-7b673987be2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:47.209513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e9752-a6fa-4e69-b06c-7b673987be2f service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:47.209513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e9752-a6fa-4e69-b06c-7b673987be2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:47.209513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e9752-a6fa-4e69-b06c-7b673987be2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:47.216761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e9752-a6fa-4e69-b06c-7b673987be2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:47.218220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e9752-a6fa-4e69-b06c-7b673987be2f service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:47.225949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1221/2426] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:26:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:26:47.309333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350fb38-93f0-4d5d-b5d0-ba6a15bece52 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:47.309620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350fb38-93f0-4d5d-b5d0-ba6a15bece52 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:47.309893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2350fb38-93f0-4d5d-b5d0-ba6a15bece52 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:47.311221 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1206/2427] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:26:47.315984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a915e2-b0a6-4e44-9f60-41890a7174e8 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:47.316278 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a915e2-b0a6-4e44-9f60-41890a7174e8 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:47.316558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a915e2-b0a6-4e44-9f60-41890a7174e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:47.336739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-91a915e2-b0a6-4e44-9f60-41890a7174e8 None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:47.337948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-91a915e2-b0a6-4e44-9f60-41890a7174e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:47.347144 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1222/2428] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:26:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:47.357587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350fb38-93f0-4d5d-b5d0-ba6a15bece52 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:47.369407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2350fb38-93f0-4d5d-b5d0-ba6a15bece52 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:47.372894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d215ac4-2ea7-492b-9c24-e766f08ab607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:47.373464 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215ac4-2ea7-492b-9c24-e766f08ab607 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:47.373736 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215ac4-2ea7-492b-9c24-e766f08ab607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:47.374005 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d215ac4-2ea7-492b-9c24-e766f08ab607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:47.384423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d215ac4-2ea7-492b-9c24-e766f08ab607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:47.385821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d215ac4-2ea7-492b-9c24-e766f08ab607 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:47.396134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1207/2429] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:26:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:26:51.817799 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b65117-fc66-4d4d-baca-331a28edbdca None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:51.818269 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b65117-fc66-4d4d-baca-331a28edbdca None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:51.818367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b65117-fc66-4d4d-baca-331a28edbdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:51.841090 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-65b65117-fc66-4d4d-baca-331a28edbdca None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:26:51.842301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-65b65117-fc66-4d4d-baca-331a28edbdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:26:51.899535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1223/2430] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:26:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:26:51.911630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d215ac4-2ea7-492b-9c24-e766f08ab607 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:26:51.937932 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:26:51.979066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d215ac4-2ea7-492b-9c24-e766f08ab607 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:26:51.984121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7e468-185c-45f2-abdd-0dd917b4bce5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:26:51.984711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e468-185c-45f2-abdd-0dd917b4bce5 service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:26:51.984983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e468-185c-45f2-abdd-0dd917b4bce5 service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:26:51.985250 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e468-185c-45f2-abdd-0dd917b4bce5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:26:51.993145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7e468-185c-45f2-abdd-0dd917b4bce5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:26:51.995619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7e468-185c-45f2-abdd-0dd917b4bce5 service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:26:52.004269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1208/2431] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:26:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:27:01.892625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=36 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:27:02.000878 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=22 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:27:17.196561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b65117-fc66-4d4d-baca-331a28edbdca None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:27:17.198002 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-65b65117-fc66-4d4d-baca-331a28edbdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:27:17.209117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b65117-fc66-4d4d-baca-331a28edbdca None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:27:17.212653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd7ab6-606b-4af4-98d5-a613733727a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:27:17.213245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd7ab6-606b-4af4-98d5-a613733727a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:27:17.213518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd7ab6-606b-4af4-98d5-a613733727a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:27:17.213787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd7ab6-606b-4af4-98d5-a613733727a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:27:17.215466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c2bd7ab6-606b-4af4-98d5-a613733727a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:27:17.228382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd7ab6-606b-4af4-98d5-a613733727a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:27:17.229739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd7ab6-606b-4af4-98d5-a613733727a5 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:27:17.241153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1224/2432] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:27:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:27:17.397308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7e468-185c-45f2-abdd-0dd917b4bce5 service swift] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:27:17.406544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7e468-185c-45f2-abdd-0dd917b4bce5 service swift] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:27:17.410139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab56a3-6368-4b75-bef2-cf52afa6c596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:27:17.410727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab56a3-6368-4b75-bef2-cf52afa6c596 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:27:17.411007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab56a3-6368-4b75-bef2-cf52afa6c596 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:27:17.411286 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab56a3-6368-4b75-bef2-cf52afa6c596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:27:17.413017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-18ab56a3-6368-4b75-bef2-cf52afa6c596 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:27:17.426725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab56a3-6368-4b75-bef2-cf52afa6c596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:27:17.428171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab56a3-6368-4b75-bef2-cf52afa6c596 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:27:17.439348 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1209/2433] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:27:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:27:24.846267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25beec32-0554-40e4-9f2a-9b1b9ff7de9a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:27:24.846861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25beec32-0554-40e4-9f2a-9b1b9ff7de9a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:27:24.847100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25beec32-0554-40e4-9f2a-9b1b9ff7de9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:27:24.867626 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-25beec32-0554-40e4-9f2a-9b1b9ff7de9a None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:27:24.868883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-25beec32-0554-40e4-9f2a-9b1b9ff7de9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:27:24.877831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1225/2434] 10.222.0.15 () {58 vars in 967 bytes} [Wed Sep 7 16:27:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:27:24.901896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab56a3-6368-4b75-bef2-cf52afa6c596 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:27:24.913117 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab56a3-6368-4b75-bef2-cf52afa6c596 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:27:24.918973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8921161-f8f8-4d2f-a43a-4981bdbaf1c7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:27:24.919565 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8921161-f8f8-4d2f-a43a-4981bdbaf1c7 service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:27:24.919869 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8921161-f8f8-4d2f-a43a-4981bdbaf1c7 service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:27:24.920142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8921161-f8f8-4d2f-a43a-4981bdbaf1c7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:27:24.928886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8921161-f8f8-4d2f-a43a-4981bdbaf1c7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:27:24.930428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8921161-f8f8-4d2f-a43a-4981bdbaf1c7 service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:27:24.939013 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1210/2435] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:27:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:27:31.624388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25beec32-0554-40e4-9f2a-9b1b9ff7de9a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:27:31.634625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25beec32-0554-40e4-9f2a-9b1b9ff7de9a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:27:31.638737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde080e-ef5f-4635-9b90-adc36c943744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:27:31.639336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde080e-ef5f-4635-9b90-adc36c943744 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:27:31.639616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde080e-ef5f-4635-9b90-adc36c943744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:27:31.639925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde080e-ef5f-4635-9b90-adc36c943744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:27:31.648839 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde080e-ef5f-4635-9b90-adc36c943744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:27:31.650876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde080e-ef5f-4635-9b90-adc36c943744 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:27:31.661787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1226/2436] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:27:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:27:34.935788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=12 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:27:41.656448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=15 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:27:55.795806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8921161-f8f8-4d2f-a43a-4981bdbaf1c7 service swift] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:27:55.797186 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e8921161-f8f8-4d2f-a43a-4981bdbaf1c7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:27:55.813103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8921161-f8f8-4d2f-a43a-4981bdbaf1c7 service swift] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:27:55.818443 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07650718-a905-42be-aa95-d282889c5f89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:27:55.819023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07650718-a905-42be-aa95-d282889c5f89 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:27:55.819297 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07650718-a905-42be-aa95-d282889c5f89 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:27:55.819568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07650718-a905-42be-aa95-d282889c5f89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:27:55.821325 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-07650718-a905-42be-aa95-d282889c5f89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:27:55.879000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07650718-a905-42be-aa95-d282889c5f89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:27:55.880616 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07650718-a905-42be-aa95-d282889c5f89 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:27:55.889556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1211/2437] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:28:05.885320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=28 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:28:15.656788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde080e-ef5f-4635-9b90-adc36c943744 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:28:15.665621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde080e-ef5f-4635-9b90-adc36c943744 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:28:15.669486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cdabb-c9b1-4b35-a398-da67659333d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:28:15.670085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cdabb-c9b1-4b35-a398-da67659333d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:28:15.670366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cdabb-c9b1-4b35-a398-da67659333d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:28:15.670644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95cdabb-c9b1-4b35-a398-da67659333d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:28:15.673036 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f95cdabb-c9b1-4b35-a398-da67659333d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:28:15.682873 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cdabb-c9b1-4b35-a398-da67659333d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:28:15.684340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95cdabb-c9b1-4b35-a398-da67659333d7 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:28:15.692389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1227/2438] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:28:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:28:25.688825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=6 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:28:34.299989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07650718-a905-42be-aa95-d282889c5f89 service nova] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:28:34.311115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07650718-a905-42be-aa95-d282889c5f89 service nova] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:28:34.317024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98c808-79d7-4928-a413-27a1417d36dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:28:34.317720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98c808-79d7-4928-a413-27a1417d36dd service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:28:34.317997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98c808-79d7-4928-a413-27a1417d36dd service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:28:34.318267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98c808-79d7-4928-a413-27a1417d36dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:28:34.320079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ae98c808-79d7-4928-a413-27a1417d36dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:28:34.377927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98c808-79d7-4928-a413-27a1417d36dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:28:34.379476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98c808-79d7-4928-a413-27a1417d36dd service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:28:34.388214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1212/2439] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:28:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:28:40.445493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cdabb-c9b1-4b35-a398-da67659333d7 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:28:40.458917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95cdabb-c9b1-4b35-a398-da67659333d7 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:28:40.464760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bd6e9-16c0-4b2e-9a48-8dad5d740dec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:28:40.465445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bd6e9-16c0-4b2e-9a48-8dad5d740dec service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:28:40.465790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bd6e9-16c0-4b2e-9a48-8dad5d740dec service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:28:40.466261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bd6e9-16c0-4b2e-9a48-8dad5d740dec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:28:40.468580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-929bd6e9-16c0-4b2e-9a48-8dad5d740dec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:28:40.481702 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bd6e9-16c0-4b2e-9a48-8dad5d740dec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:28:40.483461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bd6e9-16c0-4b2e-9a48-8dad5d740dec service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:28:40.495329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1228/2440] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:28:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:28:44.384194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=28 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:28:45.485593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748af0c-e687-4579-97c0-84a8db2a7e2d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:28:45.486163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748af0c-e687-4579-97c0-84a8db2a7e2d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:28:45.486163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748af0c-e687-4579-97c0-84a8db2a7e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:28:45.504992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-b748af0c-e687-4579-97c0-84a8db2a7e2d None None] MFA Rules not processed for user `8430cbddaa1448f6ade09a082fa29b13`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:28:45.506173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b748af0c-e687-4579-97c0-84a8db2a7e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:28:45.513046 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1213/2441] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:28:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:28:45.817544 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bd6e9-16c0-4b2e-9a48-8dad5d740dec service nova] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:28:45.826901 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bd6e9-16c0-4b2e-9a48-8dad5d740dec service nova] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:28:45.831140 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c29c2e-cb1c-4cef-9efe-8e808f5a763d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:28:45.831745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c29c2e-cb1c-4cef-9efe-8e808f5a763d service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:28:45.832056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c29c2e-cb1c-4cef-9efe-8e808f5a763d service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:28:45.832380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c29c2e-cb1c-4cef-9efe-8e808f5a763d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:28:45.834274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c8c29c2e-cb1c-4cef-9efe-8e808f5a763d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:28:45.843927 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c29c2e-cb1c-4cef-9efe-8e808f5a763d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:28:45.845353 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c29c2e-cb1c-4cef-9efe-8e808f5a763d service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:28:45.853373 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1229/2442] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:28:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:28:46.253734 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748af0c-e687-4579-97c0-84a8db2a7e2d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:28:46.257646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b748af0c-e687-4579-97c0-84a8db2a7e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:28:46.330962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748af0c-e687-4579-97c0-84a8db2a7e2d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:28:46.336224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7626b4-babf-4558-b445-5a48745cd535 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:28:46.339109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7626b4-babf-4558-b445-5a48745cd535 service placement] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:28:46.339109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7626b4-babf-4558-b445-5a48745cd535 service placement] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:28:46.339109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7626b4-babf-4558-b445-5a48745cd535 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:28:46.339109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-fc7626b4-babf-4558-b445-5a48745cd535 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:28:46.350552 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7626b4-babf-4558-b445-5a48745cd535 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:28:46.352163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7626b4-babf-4558-b445-5a48745cd535 service placement] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:28:46.363159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1214/2443] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:28:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:28:46.392210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663d40c-e6cb-4363-8a40-4135778e6506 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:28:46.392726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663d40c-e6cb-4363-8a40-4135778e6506 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:28:46.393009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663d40c-e6cb-4363-8a40-4135778e6506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:28:46.412127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-0663d40c-e6cb-4363-8a40-4135778e6506 None None] MFA Rules not processed for user `d3da95b627b74c28b921586d0da89521`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:28:46.413324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0663d40c-e6cb-4363-8a40-4135778e6506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:28:46.475515 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1230/2444] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:28:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:28:56.357829 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=35 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:28:56.468154 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=37 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:28:59.591410 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7626b4-babf-4558-b445-5a48745cd535 service placement] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:28:59.604483 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7626b4-babf-4558-b445-5a48745cd535 service placement] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:28:59.610116 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7234f8-87f2-4c07-8829-a6634e9462c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:28:59.610765 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7234f8-87f2-4c07-8829-a6634e9462c1 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:28:59.611038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7234f8-87f2-4c07-8829-a6634e9462c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:28:59.611310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7234f8-87f2-4c07-8829-a6634e9462c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:28:59.613019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-db7234f8-87f2-4c07-8829-a6634e9462c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:28:59.624939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7234f8-87f2-4c07-8829-a6634e9462c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:28:59.626496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7234f8-87f2-4c07-8829-a6634e9462c1 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:28:59.636956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1215/2445] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:28:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:29:03.794076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663d40c-e6cb-4363-8a40-4135778e6506 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:29:03.807669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663d40c-e6cb-4363-8a40-4135778e6506 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:29:03.813540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fea8c4-e741-419f-8da7-189b4f92a945 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:29:03.814144 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fea8c4-e741-419f-8da7-189b4f92a945 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:29:03.814429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fea8c4-e741-419f-8da7-189b4f92a945 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:29:03.815108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fea8c4-e741-419f-8da7-189b4f92a945 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:29:03.816949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c2fea8c4-e741-419f-8da7-189b4f92a945 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:29:03.828841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fea8c4-e741-419f-8da7-189b4f92a945 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:29:03.830425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fea8c4-e741-419f-8da7-189b4f92a945 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:29:03.841095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1231/2446] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:29:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:29:09.632582 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=6 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:29:13.836471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=6 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:29:30.885940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06924c1-9df9-46a5-b498-d76e32554781 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:29:30.886396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06924c1-9df9-46a5-b498-d76e32554781 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:29:30.886499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06924c1-9df9-46a5-b498-d76e32554781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:29:30.905830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-a06924c1-9df9-46a5-b498-d76e32554781 None None] MFA Rules not processed for user `be10df0a4d1d42208c46615d6ddd970c`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:29:30.907039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a06924c1-9df9-46a5-b498-d76e32554781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:29:30.962940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1216/2447] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:29:32.958287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb09e3-e2b3-401f-b04a-f1f9428d9bc7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:29:32.961520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb09e3-e2b3-401f-b04a-f1f9428d9bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:29:32.961520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb09e3-e2b3-401f-b04a-f1f9428d9bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:29:32.978784 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-59cb09e3-e2b3-401f-b04a-f1f9428d9bc7 None None] MFA Rules not processed for user `22c867d61b384f9e8f692b053526f1b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:29:32.980015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-59cb09e3-e2b3-401f-b04a-f1f9428d9bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:29:32.987009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1232/2448] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:29:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:29:40.956335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=25 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:29:42.974141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:29:50.169564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06924c1-9df9-46a5-b498-d76e32554781 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:29:50.170981 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a06924c1-9df9-46a5-b498-d76e32554781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:29:50.231694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06924c1-9df9-46a5-b498-d76e32554781 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:29:50.235442 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad0070-0142-4cf6-84da-f8175918ef37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:29:50.236540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad0070-0142-4cf6-84da-f8175918ef37 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:29:50.236540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad0070-0142-4cf6-84da-f8175918ef37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:29:50.237961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad0070-0142-4cf6-84da-f8175918ef37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:29:50.238549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-09ad0070-0142-4cf6-84da-f8175918ef37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:29:50.252956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad0070-0142-4cf6-84da-f8175918ef37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:29:50.254888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad0070-0142-4cf6-84da-f8175918ef37 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:29:50.264300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1217/2449] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:29:50 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:30:00.260106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=26 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:30:16.498461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb09e3-e2b3-401f-b04a-f1f9428d9bc7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:16.509082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb09e3-e2b3-401f-b04a-f1f9428d9bc7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:16.513864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec255685-e597-47da-8c9a-fa593f832464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:16.514666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec255685-e597-47da-8c9a-fa593f832464 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:16.515139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec255685-e597-47da-8c9a-fa593f832464 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:16.515456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec255685-e597-47da-8c9a-fa593f832464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:16.517234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ec255685-e597-47da-8c9a-fa593f832464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:16.529749 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec255685-e597-47da-8c9a-fa593f832464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:16.531100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec255685-e597-47da-8c9a-fa593f832464 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:16.539630 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1233/2450] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:30:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:30:16.697641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad0070-0142-4cf6-84da-f8175918ef37 service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:16.707079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad0070-0142-4cf6-84da-f8175918ef37 service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:16.710983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701b9cc-ccf6-47e9-9f2b-18396aaf8ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:16.711580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701b9cc-ccf6-47e9-9f2b-18396aaf8ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:16.711886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701b9cc-ccf6-47e9-9f2b-18396aaf8ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:16.712666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701b9cc-ccf6-47e9-9f2b-18396aaf8ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:16.714406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8701b9cc-ccf6-47e9-9f2b-18396aaf8ac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:16.777170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701b9cc-ccf6-47e9-9f2b-18396aaf8ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:16.778594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701b9cc-ccf6-47e9-9f2b-18396aaf8ac6 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:16.788078 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1218/2451] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:30:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:30:17.005867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904a1bf-56b8-44a3-bd24-ba57bc538459 None None] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:17.010306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904a1bf-56b8-44a3-bd24-ba57bc538459 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:17.010306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904a1bf-56b8-44a3-bd24-ba57bc538459 None None] PATH_INFO: `/` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:17.010306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1234/2452] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:30:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:30:17.015312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5d78c-1b44-47b6-b3df-d8b044f729eb None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:17.015596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5d78c-1b44-47b6-b3df-d8b044f729eb None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:17.015905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce5d78c-1b44-47b6-b3df-d8b044f729eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:17.038019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0ce5d78c-1b44-47b6-b3df-d8b044f729eb None None] MFA Rules not processed for user `b6a6b52462624e7f84f479144353b51e`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:30:17.039283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0ce5d78c-1b44-47b6-b3df-d8b044f729eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:17.053322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1219/2453] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:30:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:30:17.067262 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904a1bf-56b8-44a3-bd24-ba57bc538459 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:17.078192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904a1bf-56b8-44a3-bd24-ba57bc538459 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:17.082150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322d600-f8a9-464d-a333-3e8cb6212d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:17.082741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322d600-f8a9-464d-a333-3e8cb6212d2a service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:17.083015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322d600-f8a9-464d-a333-3e8cb6212d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:17.083284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322d600-f8a9-464d-a333-3e8cb6212d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:17.090949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322d600-f8a9-464d-a333-3e8cb6212d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:17.092422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322d600-f8a9-464d-a333-3e8cb6212d2a service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:17.100211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1235/2454] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:30:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:30:17.189499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce4a22-33b7-4b3f-9275-a4deb907ef47 None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:17.189815 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce4a22-33b7-4b3f-9275-a4deb907ef47 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:17.190090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ce4a22-33b7-4b3f-9275-a4deb907ef47 None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:17.191519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1220/2455] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:30:17.197482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08a9b8-a677-4a3b-b01b-57c6e6d65172 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:17.197763 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08a9b8-a677-4a3b-b01b-57c6e6d65172 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:17.198034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08a9b8-a677-4a3b-b01b-57c6e6d65172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:17.240682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8f08a9b8-a677-4a3b-b01b-57c6e6d65172 None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:30:17.241937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8f08a9b8-a677-4a3b-b01b-57c6e6d65172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:17.279630 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1236/2456] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:30:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:30:17.290687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce4a22-33b7-4b3f-9275-a4deb907ef47 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:17.302098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ce4a22-33b7-4b3f-9275-a4deb907ef47 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:17.305815 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068c895-b5e0-4ae8-b767-c7b52cffbcc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:17.306929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068c895-b5e0-4ae8-b767-c7b52cffbcc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:17.307205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068c895-b5e0-4ae8-b767-c7b52cffbcc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:17.307471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c068c895-b5e0-4ae8-b767-c7b52cffbcc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:17.318518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068c895-b5e0-4ae8-b767-c7b52cffbcc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:17.320015 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c068c895-b5e0-4ae8-b767-c7b52cffbcc0 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:17.332286 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1221/2457] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:30:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:30:20.663240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08a9b8-a677-4a3b-b01b-57c6e6d65172 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:20.667036 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8f08a9b8-a677-4a3b-b01b-57c6e6d65172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:20.736537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08a9b8-a677-4a3b-b01b-57c6e6d65172 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:20.742605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d934b-204a-4992-8b53-007a294d842a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:20.744311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d934b-204a-4992-8b53-007a294d842a service placement] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:20.744664 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d934b-204a-4992-8b53-007a294d842a service placement] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:20.745015 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d934b-204a-4992-8b53-007a294d842a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:20.746827 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5b7d934b-204a-4992-8b53-007a294d842a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:20.760126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d934b-204a-4992-8b53-007a294d842a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:20.761738 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d934b-204a-4992-8b53-007a294d842a service placement] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:20.787781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1237/2458] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:30:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:30:20.811539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068c895-b5e0-4ae8-b767-c7b52cffbcc0 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:20.812969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c068c895-b5e0-4ae8-b767-c7b52cffbcc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:20.883696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c068c895-b5e0-4ae8-b767-c7b52cffbcc0 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:20.888537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fa355-97b1-442e-94a8-ee12674c7908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:20.889159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fa355-97b1-442e-94a8-ee12674c7908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:20.889434 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fa355-97b1-442e-94a8-ee12674c7908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:20.889703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fa355-97b1-442e-94a8-ee12674c7908 admin admin] PATH_INFO: `/v3/users/d3da95b627b74c28b921586d0da89521` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:20.890823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fa355-97b1-442e-94a8-ee12674c7908 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3da95b627b74c28b921586d0da89521)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:20.892785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fa355-97b1-442e-94a8-ee12674c7908 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:21.103911 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-757fa355-97b1-442e-94a8-ee12674c7908 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3da95b627b74c28b921586d0da89521'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:30:21.179671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-757fa355-97b1-442e-94a8-ee12674c7908 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3da95b627b74c28b921586d0da89521'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:30:21.180321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-757fa355-97b1-442e-94a8-ee12674c7908 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3da95b627b74c28b921586d0da89521'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:30:21.180989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-757fa355-97b1-442e-94a8-ee12674c7908 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3da95b627b74c28b921586d0da89521'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:30:21.188716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-757fa355-97b1-442e-94a8-ee12674c7908 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3da95b627b74c28b921586d0da89521'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:30:21.206818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1222/2459] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:30:20 2022] DELETE /identity/v3/users/d3da95b627b74c28b921586d0da89521 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:30:21.460538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d934b-204a-4992-8b53-007a294d842a service placement] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:21.479248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d934b-204a-4992-8b53-007a294d842a service placement] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:21.538462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:21.539099 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:21.539380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:21.539657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] PATH_INFO: `/v3/projects/04611beea2e54fa0be05a5eaf4715323` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:21.540834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04611beea2e54fa0be05a5eaf4715323)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:21.542730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:21.675917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:30:21.682940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04611beea2e54fa0be05a5eaf4715323', 'request_id': 'req-a15c80cc-651f-4368-8d1e-8abeace8bb33'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:30:21.688767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04611beea2e54fa0be05a5eaf4715323', 'request_id': 'req-a15c80cc-651f-4368-8d1e-8abeace8bb33'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:30:21.745155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04611beea2e54fa0be05a5eaf4715323', 'request_id': 'req-a15c80cc-651f-4368-8d1e-8abeace8bb33'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:30:21.757062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] The token cache is being invalidate because project 04611beea2e54fa0be05a5eaf4715323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:30:21.757537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:30:21.758087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a15c80cc-651f-4368-8d1e-8abeace8bb33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:30:21.760169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1238/2460] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:30:21 2022] DELETE /identity/v3/projects/04611beea2e54fa0be05a5eaf4715323 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:30:21.765483 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914931e-6497-4f87-b0a0-ec88eb2a9277 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:21.765914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914931e-6497-4f87-b0a0-ec88eb2a9277 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:21.766286 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914931e-6497-4f87-b0a0-ec88eb2a9277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:21.798594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:30:21.822700 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-2914931e-6497-4f87-b0a0-ec88eb2a9277 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:30:21.824009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2914931e-6497-4f87-b0a0-ec88eb2a9277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:21.886785 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1223/2461] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:30:21.900844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba7eca-a909-43ca-b1cd-8a95c392ea86 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:21.901139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba7eca-a909-43ca-b1cd-8a95c392ea86 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:21.901445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba7eca-a909-43ca-b1cd-8a95c392ea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:21.922409 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-c8ba7eca-a909-43ca-b1cd-8a95c392ea86 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:30:21.923638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c8ba7eca-a909-43ca-b1cd-8a95c392ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:21.932276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1239/2462] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:30:21.937089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914931e-6497-4f87-b0a0-ec88eb2a9277 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:21.948232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914931e-6497-4f87-b0a0-ec88eb2a9277 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:21.953135 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136242cc-450b-4bba-b833-49322d324550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:21.953751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136242cc-450b-4bba-b833-49322d324550 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:21.954026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136242cc-450b-4bba-b833-49322d324550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:21.954666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136242cc-450b-4bba-b833-49322d324550 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:21.954928 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136242cc-450b-4bba-b833-49322d324550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:30:21.955237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136242cc-450b-4bba-b833-49322d324550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:21.956896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136242cc-450b-4bba-b833-49322d324550 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:21.969192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1224/2463] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:30:21.977724 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba7eca-a909-43ca-b1cd-8a95c392ea86 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:21.983980 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba7eca-a909-43ca-b1cd-8a95c392ea86 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:21.988807 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab221e-bd00-4375-822d-1559353a9e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:21.989447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab221e-bd00-4375-822d-1559353a9e20 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:21.989722 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab221e-bd00-4375-822d-1559353a9e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:21.989991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab221e-bd00-4375-822d-1559353a9e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:21.990644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab221e-bd00-4375-822d-1559353a9e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:21.992591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab221e-bd00-4375-822d-1559353a9e20 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:22.089199 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1240/2464] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:30:21 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:30:22.093418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136242cc-450b-4bba-b833-49322d324550 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:22.103357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136242cc-450b-4bba-b833-49322d324550 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:22.158444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da471b8-82b5-4cfb-acd7-d15e20832b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:22.159049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da471b8-82b5-4cfb-acd7-d15e20832b88 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:22.159333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da471b8-82b5-4cfb-acd7-d15e20832b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:22.159609 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da471b8-82b5-4cfb-acd7-d15e20832b88 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:22.160263 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da471b8-82b5-4cfb-acd7-d15e20832b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:22.162171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da471b8-82b5-4cfb-acd7-d15e20832b88 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:22.242075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1225/2465] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:30:22 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:30:22.246794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab221e-bd00-4375-822d-1559353a9e20 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:22.257254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab221e-bd00-4375-822d-1559353a9e20 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:22.261837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b9968-7b92-448f-982b-b606395167a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:22.268853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b9968-7b92-448f-982b-b606395167a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:22.268853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b9968-7b92-448f-982b-b606395167a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:22.268853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b9968-7b92-448f-982b-b606395167a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:22.268853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b9968-7b92-448f-982b-b606395167a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:22.268853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b9968-7b92-448f-982b-b606395167a8 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:22.274895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1241/2466] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:30:22.278717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da471b8-82b5-4cfb-acd7-d15e20832b88 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:22.288316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da471b8-82b5-4cfb-acd7-d15e20832b88 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:22.293343 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132b8f8-eec7-485a-8f1d-91cb4ba6de8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:22.293928 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132b8f8-eec7-485a-8f1d-91cb4ba6de8f admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:22.294208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132b8f8-eec7-485a-8f1d-91cb4ba6de8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:22.294482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132b8f8-eec7-485a-8f1d-91cb4ba6de8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:22.295111 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132b8f8-eec7-485a-8f1d-91cb4ba6de8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:22.296720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132b8f8-eec7-485a-8f1d-91cb4ba6de8f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:22.307525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1226/2467] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:30:22.311814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b9968-7b92-448f-982b-b606395167a8 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:22.321862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b9968-7b92-448f-982b-b606395167a8 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:22.326447 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189ec98-72b9-4eda-ace3-4decc22a029f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:22.327083 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189ec98-72b9-4eda-ace3-4decc22a029f admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:22.327362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189ec98-72b9-4eda-ace3-4decc22a029f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:22.327636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189ec98-72b9-4eda-ace3-4decc22a029f admin admin] PATH_INFO: `/v3/projects/b7b0bd00008a4ceaa2979a6ec6a10dbd/users/537431860e3d4c4fa629254f8c4c0cf4/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:22.342370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189ec98-72b9-4eda-ace3-4decc22a029f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7b0bd00008a4ceaa2979a6ec6a10dbd, user_id=537431860e3d4c4fa629254f8c4c0cf4, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:22.345115 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189ec98-72b9-4eda-ace3-4decc22a029f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:22.430482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1242/2468] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:30:22 2022] PUT /identity/v3/projects/b7b0bd00008a4ceaa2979a6ec6a10dbd/users/537431860e3d4c4fa629254f8c4c0cf4/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:30:22.436735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0892654-8a16-42b8-8eb3-9a8f31526bca None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:22.437086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0892654-8a16-42b8-8eb3-9a8f31526bca None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:22.437460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0892654-8a16-42b8-8eb3-9a8f31526bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:22.456115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-a0892654-8a16-42b8-8eb3-9a8f31526bca None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:30:22.457312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a0892654-8a16-42b8-8eb3-9a8f31526bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:22.516710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1227/2469] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:30:22.526845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189ec98-72b9-4eda-ace3-4decc22a029f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:22.528630 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0189ec98-72b9-4eda-ace3-4decc22a029f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:22.585063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189ec98-72b9-4eda-ace3-4decc22a029f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:22.589243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf881c-b798-4cf2-8826-c366a59cbd8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:22.589843 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf881c-b798-4cf2-8826-c366a59cbd8d service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:22.590141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf881c-b798-4cf2-8826-c366a59cbd8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:22.590414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf881c-b798-4cf2-8826-c366a59cbd8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:22.599231 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf881c-b798-4cf2-8826-c366a59cbd8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:22.600682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bf881c-b798-4cf2-8826-c366a59cbd8d service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:22.610546 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1243/2470] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:30:28.766628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c59258-87c7-441e-b87c-08015070bf55 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:28.767105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c59258-87c7-441e-b87c-08015070bf55 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:28.767256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c59258-87c7-441e-b87c-08015070bf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:28.774522 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:30:28.799121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-48c59258-87c7-441e-b87c-08015070bf55 None None] MFA Rules not processed for user `537431860e3d4c4fa629254f8c4c0cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:30:28.800414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-48c59258-87c7-441e-b87c-08015070bf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:28.880470 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1228/2471] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:30:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:30:28.895679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf881c-b798-4cf2-8826-c366a59cbd8d service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:28.897220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-36bf881c-b798-4cf2-8826-c366a59cbd8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:28.968139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:30:29.013768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf881c-b798-4cf2-8826-c366a59cbd8d service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:29.019787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775ac97-c116-4508-883d-70ec07dc87ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:29.021030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775ac97-c116-4508-883d-70ec07dc87ad service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:29.021030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775ac97-c116-4508-883d-70ec07dc87ad service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:29.021030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775ac97-c116-4508-883d-70ec07dc87ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:29.030800 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775ac97-c116-4508-883d-70ec07dc87ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:29.032960 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775ac97-c116-4508-883d-70ec07dc87ad service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:29.044078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1244/2472] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:30:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:30:32.132500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ec96b-1a4a-4873-9ff4-a1d398a61bf0 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:32.133408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ec96b-1a4a-4873-9ff4-a1d398a61bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:32.133845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ec96b-1a4a-4873-9ff4-a1d398a61bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:32.160099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-4d9ec96b-1a4a-4873-9ff4-a1d398a61bf0 None None] MFA Rules not processed for user `537431860e3d4c4fa629254f8c4c0cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:30:32.161017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4d9ec96b-1a4a-4873-9ff4-a1d398a61bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:32.176892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1229/2473] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:30:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:30:35.161006 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775ac97-c116-4508-883d-70ec07dc87ad service nova] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:35.162463 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d775ac97-c116-4508-883d-70ec07dc87ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:35.236189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775ac97-c116-4508-883d-70ec07dc87ad service nova] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:35.241319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ff39d-ad89-4c8b-86ce-8ea5f49c9c31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:35.241877 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ff39d-ad89-4c8b-86ce-8ea5f49c9c31 service placement] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:35.242161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ff39d-ad89-4c8b-86ce-8ea5f49c9c31 service placement] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:35.242533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ff39d-ad89-4c8b-86ce-8ea5f49c9c31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:35.244730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8a8ff39d-ad89-4c8b-86ce-8ea5f49c9c31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:35.256846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ff39d-ad89-4c8b-86ce-8ea5f49c9c31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:35.258431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ff39d-ad89-4c8b-86ce-8ea5f49c9c31 service placement] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:35.270963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1245/2474] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:30:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:30:42.149384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=34 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:30:44.424124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489466a0-5974-4f3f-8746-7bebd168c9e5 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:44.425102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489466a0-5974-4f3f-8746-7bebd168c9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:44.429371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489466a0-5974-4f3f-8746-7bebd168c9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:44.467355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-489466a0-5974-4f3f-8746-7bebd168c9e5 None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:30:44.468596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-489466a0-5974-4f3f-8746-7bebd168c9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:44.523596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1230/2475] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:30:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:30:44.654321 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ff39d-ad89-4c8b-86ce-8ea5f49c9c31 service placement] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:30:44.654744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8a8ff39d-ad89-4c8b-86ce-8ea5f49c9c31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:30:44.723550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ff39d-ad89-4c8b-86ce-8ea5f49c9c31 service placement] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:30:44.729415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb2f93-236b-4a39-8273-95ebbddb0e5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:30:44.730030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb2f93-236b-4a39-8273-95ebbddb0e5d service swift] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:30:44.730314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb2f93-236b-4a39-8273-95ebbddb0e5d service swift] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:30:44.730593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb2f93-236b-4a39-8273-95ebbddb0e5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:30:44.738628 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb2f93-236b-4a39-8273-95ebbddb0e5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:30:44.740262 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb2f93-236b-4a39-8273-95ebbddb0e5d service swift] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:30:44.748226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1246/2476] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:30:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:30:54.516654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=29 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:30:54.745056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=80 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:31:02.842701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f316c5-ec05-4946-ad3c-a49c3eedd67a None None] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:31:02.843107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f316c5-ec05-4946-ad3c-a49c3eedd67a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:31:02.843256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f316c5-ec05-4946-ad3c-a49c3eedd67a None None] PATH_INFO: `/` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:31:02.844643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1231/2477] 10.222.0.15 () {54 vars in 921 bytes} [Wed Sep 7 16:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:31:02.849565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768228a-93f9-419d-a7d5-96afc0b34122 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:31:02.850078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768228a-93f9-419d-a7d5-96afc0b34122 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:31:02.850251 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768228a-93f9-419d-a7d5-96afc0b34122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:31:02.870825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-9768228a-93f9-419d-a7d5-96afc0b34122 None None] MFA Rules not processed for user `45061db68b1e40f59935cc9a46d2ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:31:02.872289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9768228a-93f9-419d-a7d5-96afc0b34122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:31:02.881862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1247/2478] 10.222.0.15 () {58 vars in 1018 bytes} [Wed Sep 7 16:31:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:31:02.894873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f316c5-ec05-4946-ad3c-a49c3eedd67a None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:31:02.896074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-56f316c5-ec05-4946-ad3c-a49c3eedd67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:31:02.955017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f316c5-ec05-4946-ad3c-a49c3eedd67a None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:31:02.958539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26c659-26bd-4ff2-861f-2dc2ee15035d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5d7cf7534941cb857a4ab5867d978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:31:02.959185 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26c659-26bd-4ff2-861f-2dc2ee15035d service glance] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:31:02.959465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26c659-26bd-4ff2-861f-2dc2ee15035d service glance] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:31:02.959738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26c659-26bd-4ff2-861f-2dc2ee15035d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:31:02.970316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26c659-26bd-4ff2-861f-2dc2ee15035d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:31:02.971778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26c659-26bd-4ff2-861f-2dc2ee15035d service glance] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:31:02.983353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1232/2479] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:31:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:31:06.003029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a7320-c27a-406f-8a28-25e1024fcb2d None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:31:06.003525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a7320-c27a-406f-8a28-25e1024fcb2d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:31:06.003642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a7320-c27a-406f-8a28-25e1024fcb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:31:06.022937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-9e3a7320-c27a-406f-8a28-25e1024fcb2d None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:31:06.024170 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9e3a7320-c27a-406f-8a28-25e1024fcb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:31:06.031189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1248/2480] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:31:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:31:06.043452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26c659-26bd-4ff2-861f-2dc2ee15035d service glance] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:31:06.054327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26c659-26bd-4ff2-861f-2dc2ee15035d service glance] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:31:06.059909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e293770-6f6d-4a1e-86af-eda556ffe204 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:31:06.060490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e293770-6f6d-4a1e-86af-eda556ffe204 service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:31:06.060766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e293770-6f6d-4a1e-86af-eda556ffe204 service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:31:06.061037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e293770-6f6d-4a1e-86af-eda556ffe204 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:31:06.069060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e293770-6f6d-4a1e-86af-eda556ffe204 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:31:06.070797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e293770-6f6d-4a1e-86af-eda556ffe204 service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:31:06.078666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1233/2481] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:31:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:31:16.017719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=6 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:31:16.076160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=32 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:31:29.417957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee03a6-e279-4f8e-86d9-1f9740173ebf None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:31:29.418462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee03a6-e279-4f8e-86d9-1f9740173ebf None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:31:29.418565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee03a6-e279-4f8e-86d9-1f9740173ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:31:29.440496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-c8ee03a6-e279-4f8e-86d9-1f9740173ebf None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:31:29.441716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c8ee03a6-e279-4f8e-86d9-1f9740173ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:31:29.449500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1249/2482] 10.222.0.15 () {58 vars in 967 bytes} [Wed Sep 7 16:31:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:31:29.461760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e293770-6f6d-4a1e-86af-eda556ffe204 service swift] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:31:29.472815 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e293770-6f6d-4a1e-86af-eda556ffe204 service swift] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:31:29.478418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17916f8-c4ce-43f1-9bc9-a5aec95f850c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:31:29.478993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17916f8-c4ce-43f1-9bc9-a5aec95f850c service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:31:29.479268 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17916f8-c4ce-43f1-9bc9-a5aec95f850c service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:31:29.479539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17916f8-c4ce-43f1-9bc9-a5aec95f850c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:31:29.487512 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17916f8-c4ce-43f1-9bc9-a5aec95f850c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:31:29.489096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17916f8-c4ce-43f1-9bc9-a5aec95f850c service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:31:29.497413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1234/2483] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:31:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 16:31:39.433563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:31:39.493722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=6 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:31:58.064438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee03a6-e279-4f8e-86d9-1f9740173ebf None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:31:58.083347 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee03a6-e279-4f8e-86d9-1f9740173ebf None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:31:58.086928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80373-9f71-4f94-8af8-c84da1acb52d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:31:58.087520 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80373-9f71-4f94-8af8-c84da1acb52d service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:31:58.087832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80373-9f71-4f94-8af8-c84da1acb52d service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:31:58.088110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af80373-9f71-4f94-8af8-c84da1acb52d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:31:58.089797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6af80373-9f71-4f94-8af8-c84da1acb52d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:31:58.101902 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af80373-9f71-4f94-8af8-c84da1acb52d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:31:58.103309 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af80373-9f71-4f94-8af8-c84da1acb52d service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:31:58.112864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1250/2484] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:01.394101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17916f8-c4ce-43f1-9bc9-a5aec95f850c service swift] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:01.395637 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e17916f8-c4ce-43f1-9bc9-a5aec95f850c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:01.410577 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17916f8-c4ce-43f1-9bc9-a5aec95f850c service swift] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:01.416151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d4bd3-e906-4b3c-a1a4-3a639c543ed3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:01.416744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d4bd3-e906-4b3c-a1a4-3a639c543ed3 service placement] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:01.417025 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d4bd3-e906-4b3c-a1a4-3a639c543ed3 service placement] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:01.417304 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d4bd3-e906-4b3c-a1a4-3a639c543ed3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:01.419050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9d6d4bd3-e906-4b3c-a1a4-3a639c543ed3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:01.431529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d4bd3-e906-4b3c-a1a4-3a639c543ed3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:01.433251 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d4bd3-e906-4b3c-a1a4-3a639c543ed3 service placement] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:01.444218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1235/2485] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:01.495503 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80373-9f71-4f94-8af8-c84da1acb52d service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:01.496926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6af80373-9f71-4f94-8af8-c84da1acb52d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:01.510670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af80373-9f71-4f94-8af8-c84da1acb52d service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:01.515420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863e487-8b85-40bb-a768-095dc190545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:01.516117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863e487-8b85-40bb-a768-095dc190545c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:01.516399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863e487-8b85-40bb-a768-095dc190545c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:01.516675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863e487-8b85-40bb-a768-095dc190545c admin admin] PATH_INFO: `/v3/users/22c867d61b384f9e8f692b053526f1b3` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:01.517790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863e487-8b85-40bb-a768-095dc190545c admin admin] RBAC: Authorizing `identity:delete_user(user_id=22c867d61b384f9e8f692b053526f1b3)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:01.519751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863e487-8b85-40bb-a768-095dc190545c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:01.685545 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-b863e487-8b85-40bb-a768-095dc190545c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22c867d61b384f9e8f692b053526f1b3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:01.757031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-b863e487-8b85-40bb-a768-095dc190545c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22c867d61b384f9e8f692b053526f1b3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:01.757659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-b863e487-8b85-40bb-a768-095dc190545c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22c867d61b384f9e8f692b053526f1b3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:01.758285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-b863e487-8b85-40bb-a768-095dc190545c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22c867d61b384f9e8f692b053526f1b3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:01.766683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-b863e487-8b85-40bb-a768-095dc190545c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22c867d61b384f9e8f692b053526f1b3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:01.789797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1251/2486] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:01 2022] DELETE /identity/v3/users/22c867d61b384f9e8f692b053526f1b3 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:02.062068 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d4bd3-e906-4b3c-a1a4-3a639c543ed3 service placement] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:02.072981 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d4bd3-e906-4b3c-a1a4-3a639c543ed3 service placement] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:02.129726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:02.130380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:02.130664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:02.130939 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] PATH_INFO: `/v3/projects/7170bcb816ec44899c533c914d861d8f` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:02.132094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7170bcb816ec44899c533c914d861d8f)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:02.133973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:02.249664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7170bcb816ec44899c533c914d861d8f', 'request_id': 'req-705b2906-c28e-4e4e-bd63-65de4efcffa7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:02.322924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7170bcb816ec44899c533c914d861d8f', 'request_id': 'req-705b2906-c28e-4e4e-bd63-65de4efcffa7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:02.330043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7170bcb816ec44899c533c914d861d8f', 'request_id': 'req-705b2906-c28e-4e4e-bd63-65de4efcffa7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:02.340303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] The token cache is being invalidate because project 7170bcb816ec44899c533c914d861d8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:02.344017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:02.344017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-705b2906-c28e-4e4e-bd63-65de4efcffa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:02.344017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1236/2487] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:02 2022] DELETE /identity/v3/projects/7170bcb816ec44899c533c914d861d8f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:02.347674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a104-eb06-4e79-8e87-840c5c09e95a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:02.347988 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a104-eb06-4e79-8e87-840c5c09e95a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:02.348307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61a104-eb06-4e79-8e87-840c5c09e95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:02.367844 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6e61a104-eb06-4e79-8e87-840c5c09e95a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:02.369120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6e61a104-eb06-4e79-8e87-840c5c09e95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:02.427755 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1252/2488] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:02.435592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4ea6f-6930-426b-8eb2-36c14e67d2dd None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:02.436082 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4ea6f-6930-426b-8eb2-36c14e67d2dd None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:02.436179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4ea6f-6930-426b-8eb2-36c14e67d2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:02.455834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-31e4ea6f-6930-426b-8eb2-36c14e67d2dd None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:02.457032 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-31e4ea6f-6930-426b-8eb2-36c14e67d2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:02.464969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1237/2489] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:02.469516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61a104-eb06-4e79-8e87-840c5c09e95a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:02.479121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61a104-eb06-4e79-8e87-840c5c09e95a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:02.484245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72296a-4bb2-4d26-835e-1cbb12f0e85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:02.484832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72296a-4bb2-4d26-835e-1cbb12f0e85c admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:02.485108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72296a-4bb2-4d26-835e-1cbb12f0e85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:02.485379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72296a-4bb2-4d26-835e-1cbb12f0e85c admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:02.486037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72296a-4bb2-4d26-835e-1cbb12f0e85c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:32:02.486336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72296a-4bb2-4d26-835e-1cbb12f0e85c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:02.489178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72296a-4bb2-4d26-835e-1cbb12f0e85c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:02.500945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1253/2490] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:32:02.505219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4ea6f-6930-426b-8eb2-36c14e67d2dd None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:02.514418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4ea6f-6930-426b-8eb2-36c14e67d2dd None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:02.519074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75a4d3-df31-41f9-b5e8-cab51c43f805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:02.519665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75a4d3-df31-41f9-b5e8-cab51c43f805 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:02.519983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75a4d3-df31-41f9-b5e8-cab51c43f805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:02.520261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f75a4d3-df31-41f9-b5e8-cab51c43f805 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:02.520901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75a4d3-df31-41f9-b5e8-cab51c43f805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:02.522793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f75a4d3-df31-41f9-b5e8-cab51c43f805 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:02.593297 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1238/2491] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:32:02 2022] POST /identity/v3/projects => generated 401 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:02.597507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72296a-4bb2-4d26-835e-1cbb12f0e85c admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:02.607655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72296a-4bb2-4d26-835e-1cbb12f0e85c admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:02.641134 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:02.664566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1865f1-e045-4976-898d-eec33bbff2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:02.665171 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1865f1-e045-4976-898d-eec33bbff2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:02.665452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1865f1-e045-4976-898d-eec33bbff2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:02.665731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1865f1-e045-4976-898d-eec33bbff2f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:02.666361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1865f1-e045-4976-898d-eec33bbff2f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:02.668330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1865f1-e045-4976-898d-eec33bbff2f6 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:02.744858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1254/2492] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:32:02 2022] POST /identity/v3/users => generated 301 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:02.749293 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75a4d3-df31-41f9-b5e8-cab51c43f805 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:02.759799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f75a4d3-df31-41f9-b5e8-cab51c43f805 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:02.764592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424baa2e-6d07-4dc6-bb16-8a0160e00afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:02.765189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424baa2e-6d07-4dc6-bb16-8a0160e00afd admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:02.765481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424baa2e-6d07-4dc6-bb16-8a0160e00afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:02.765750 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424baa2e-6d07-4dc6-bb16-8a0160e00afd admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:02.766437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424baa2e-6d07-4dc6-bb16-8a0160e00afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:02.768069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424baa2e-6d07-4dc6-bb16-8a0160e00afd admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:02.777893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1239/2493] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:02.781970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1865f1-e045-4976-898d-eec33bbff2f6 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:02.791886 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1865f1-e045-4976-898d-eec33bbff2f6 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:02.797720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f4b4f-fe2f-4b0b-a28f-c018d2d6cf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:02.798342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f4b4f-fe2f-4b0b-a28f-c018d2d6cf37 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:02.798618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f4b4f-fe2f-4b0b-a28f-c018d2d6cf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:02.798888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f4b4f-fe2f-4b0b-a28f-c018d2d6cf37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:02.799521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f4b4f-fe2f-4b0b-a28f-c018d2d6cf37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:02.801162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f4b4f-fe2f-4b0b-a28f-c018d2d6cf37 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:02.811616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1255/2494] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:02.816105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424baa2e-6d07-4dc6-bb16-8a0160e00afd admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:02.826090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424baa2e-6d07-4dc6-bb16-8a0160e00afd admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:02.830793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b779cf-8d7b-4da2-be39-0cf6387f6853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:02.831482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b779cf-8d7b-4da2-be39-0cf6387f6853 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:02.831808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b779cf-8d7b-4da2-be39-0cf6387f6853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:02.832091 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b779cf-8d7b-4da2-be39-0cf6387f6853 admin admin] PATH_INFO: `/v3/projects/f6c43b2ef21748979caa05522db599a4/users/d3b1e77e07e44af7bc56a3c015e16ecc/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:02.846396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b779cf-8d7b-4da2-be39-0cf6387f6853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c43b2ef21748979caa05522db599a4, user_id=d3b1e77e07e44af7bc56a3c015e16ecc, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:02.849061 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b779cf-8d7b-4da2-be39-0cf6387f6853 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:02.916796 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1240/2495] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:02 2022] PUT /identity/v3/projects/f6c43b2ef21748979caa05522db599a4/users/d3b1e77e07e44af7bc56a3c015e16ecc/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:02.928784 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef4275-dff5-4fad-a5e3-9a0ad7a06cc7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:02.929069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef4275-dff5-4fad-a5e3-9a0ad7a06cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:02.929343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef4275-dff5-4fad-a5e3-9a0ad7a06cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:02.958533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-fdef4275-dff5-4fad-a5e3-9a0ad7a06cc7 None None] MFA Rules not processed for user `d3b1e77e07e44af7bc56a3c015e16ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:02.959746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fdef4275-dff5-4fad-a5e3-9a0ad7a06cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:03.025577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1256/2496] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:02 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:03.030828 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b779cf-8d7b-4da2-be39-0cf6387f6853 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:03.044402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:03.092841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b779cf-8d7b-4da2-be39-0cf6387f6853 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:03.097845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c47199-23e5-4bdc-8901-56a64bc74932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:03.098444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c47199-23e5-4bdc-8901-56a64bc74932 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:03.098722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c47199-23e5-4bdc-8901-56a64bc74932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:03.098998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c47199-23e5-4bdc-8901-56a64bc74932 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:03.099635 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c47199-23e5-4bdc-8901-56a64bc74932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:03.101560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c47199-23e5-4bdc-8901-56a64bc74932 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:03.221166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1241/2497] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:32:03 2022] POST /identity/v3/projects => generated 405 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:03.225186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef4275-dff5-4fad-a5e3-9a0ad7a06cc7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:03.236012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef4275-dff5-4fad-a5e3-9a0ad7a06cc7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:03.291707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d2c88-fc2e-4bfe-9593-953a0c612fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:03.292276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d2c88-fc2e-4bfe-9593-953a0c612fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:03.292556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d2c88-fc2e-4bfe-9593-953a0c612fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:03.292832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d2c88-fc2e-4bfe-9593-953a0c612fe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:03.293468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d2c88-fc2e-4bfe-9593-953a0c612fe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:03.295962 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d2c88-fc2e-4bfe-9593-953a0c612fe5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:03.402848 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1257/2498] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:32:03 2022] POST /identity/v3/users => generated 309 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:03.407468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c47199-23e5-4bdc-8901-56a64bc74932 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:03.417447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c47199-23e5-4bdc-8901-56a64bc74932 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:03.422002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe2996-bac0-4f60-8abc-e2aa487792ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:03.422582 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe2996-bac0-4f60-8abc-e2aa487792ea admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:03.422865 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe2996-bac0-4f60-8abc-e2aa487792ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:03.423133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe2996-bac0-4f60-8abc-e2aa487792ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:03.423782 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe2996-bac0-4f60-8abc-e2aa487792ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:03.425397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe2996-bac0-4f60-8abc-e2aa487792ea admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:03.435359 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1242/2499] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:03.439883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d2c88-fc2e-4bfe-9593-953a0c612fe5 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:03.450207 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d2c88-fc2e-4bfe-9593-953a0c612fe5 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:03.455342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f0660-ff3a-42df-be70-b46b40df6b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:03.456018 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f0660-ff3a-42df-be70-b46b40df6b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:03.456297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f0660-ff3a-42df-be70-b46b40df6b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:03.456594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f0660-ff3a-42df-be70-b46b40df6b39 admin admin] PATH_INFO: `/v3/projects/c9c9252d92ca497b8812d0bbe13ad027/users/f13abfbd293f435195000e1dbac9d873/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:03.471518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f0660-ff3a-42df-be70-b46b40df6b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c9252d92ca497b8812d0bbe13ad027, user_id=f13abfbd293f435195000e1dbac9d873, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:03.474175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f0660-ff3a-42df-be70-b46b40df6b39 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:03.545488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1258/2500] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:03 2022] PUT /identity/v3/projects/c9c9252d92ca497b8812d0bbe13ad027/users/f13abfbd293f435195000e1dbac9d873/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:03.549364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe2996-bac0-4f60-8abc-e2aa487792ea admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:03.559610 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe2996-bac0-4f60-8abc-e2aa487792ea admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:03.615714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c49a37-4149-41d6-88a1-ff7363ed84ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:03.616349 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c49a37-4149-41d6-88a1-ff7363ed84ff admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:03.616630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c49a37-4149-41d6-88a1-ff7363ed84ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:03.616902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c49a37-4149-41d6-88a1-ff7363ed84ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:03.617531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c49a37-4149-41d6-88a1-ff7363ed84ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:03.619202 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c49a37-4149-41d6-88a1-ff7363ed84ff admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:03.629409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1243/2501] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:03.633584 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f0660-ff3a-42df-be70-b46b40df6b39 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:03.643941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f0660-ff3a-42df-be70-b46b40df6b39 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:03.648444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3cf4f-4ac4-4bb5-8ee4-d8399738b1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:03.649079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3cf4f-4ac4-4bb5-8ee4-d8399738b1be admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:03.649355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3cf4f-4ac4-4bb5-8ee4-d8399738b1be admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:03.649627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3cf4f-4ac4-4bb5-8ee4-d8399738b1be admin admin] PATH_INFO: `/v3/projects/c9c9252d92ca497b8812d0bbe13ad027/users/f13abfbd293f435195000e1dbac9d873/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:03.651517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3cf4f-4ac4-4bb5-8ee4-d8399738b1be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9c9252d92ca497b8812d0bbe13ad027, user_id=f13abfbd293f435195000e1dbac9d873, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:03.654297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3cf4f-4ac4-4bb5-8ee4-d8399738b1be admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:03.659561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:03.664646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1259/2502] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:03 2022] PUT /identity/v3/projects/c9c9252d92ca497b8812d0bbe13ad027/users/f13abfbd293f435195000e1dbac9d873/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:03.679580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388f16b-bdef-4d33-b216-20f82c6cb9aa None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:03.679908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388f16b-bdef-4d33-b216-20f82c6cb9aa None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:03.680184 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388f16b-bdef-4d33-b216-20f82c6cb9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:03.708069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-5388f16b-bdef-4d33-b216-20f82c6cb9aa None None] MFA Rules not processed for user `f13abfbd293f435195000e1dbac9d873`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:03.709269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-5388f16b-bdef-4d33-b216-20f82c6cb9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:03.776308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1244/2503] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:03.787401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3cf4f-4ac4-4bb5-8ee4-d8399738b1be admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:03.789189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d5d3cf4f-4ac4-4bb5-8ee4-d8399738b1be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:03.866863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3cf4f-4ac4-4bb5-8ee4-d8399738b1be admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:03.870569 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc39e4c-1917-4c62-aa13-556bf479a939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:03.871169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc39e4c-1917-4c62-aa13-556bf479a939 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:03.871448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc39e4c-1917-4c62-aa13-556bf479a939 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:03.871725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc39e4c-1917-4c62-aa13-556bf479a939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:03.880518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc39e4c-1917-4c62-aa13-556bf479a939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:03.881930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc39e4c-1917-4c62-aa13-556bf479a939 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:03.891437 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1260/2504] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:03 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:04.041118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388f16b-bdef-4d33-b216-20f82c6cb9aa None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:04.051428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:04.103566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388f16b-bdef-4d33-b216-20f82c6cb9aa None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:04.108668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:04.109291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:04.109570 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:04.109842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] PATH_INFO: `/v3/users/d3b1e77e07e44af7bc56a3c015e16ecc` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:04.110956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b1e77e07e44af7bc56a3c015e16ecc)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:04.113121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:04.300779 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b1e77e07e44af7bc56a3c015e16ecc'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:04.367266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b1e77e07e44af7bc56a3c015e16ecc'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:04.367908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b1e77e07e44af7bc56a3c015e16ecc'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:04.368537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b1e77e07e44af7bc56a3c015e16ecc'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:04.375768 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b1e77e07e44af7bc56a3c015e16ecc'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:04.393817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1245/2505] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:04 2022] DELETE /identity/v3/users/d3b1e77e07e44af7bc56a3c015e16ecc => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:04.398413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31efa0-2f50-45a5-918a-60390c9aa054 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:04.398692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31efa0-2f50-45a5-918a-60390c9aa054 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:04.398960 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31efa0-2f50-45a5-918a-60390c9aa054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:04.419100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-0a31efa0-2f50-45a5-918a-60390c9aa054 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:04.420373 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0a31efa0-2f50-45a5-918a-60390c9aa054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:04.478033 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1261/2506] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:04.487728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:04.489154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:04.564246 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294af42-f5bd-430c-b044-d7129816354f admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:04.567801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae392cf-6f07-4f83-ad49-4d8a8d4dbc78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:04.568394 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae392cf-6f07-4f83-ad49-4d8a8d4dbc78 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:04.568674 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae392cf-6f07-4f83-ad49-4d8a8d4dbc78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:04.568947 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae392cf-6f07-4f83-ad49-4d8a8d4dbc78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:04.578283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae392cf-6f07-4f83-ad49-4d8a8d4dbc78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:04.579689 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae392cf-6f07-4f83-ad49-4d8a8d4dbc78 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:04.588990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1246/2507] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:04.992866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31efa0-2f50-45a5-918a-60390c9aa054 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:04.995991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0a31efa0-2f50-45a5-918a-60390c9aa054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:05.008621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31efa0-2f50-45a5-918a-60390c9aa054 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:05.013601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:05.014246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:05.014526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:05.014802 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] PATH_INFO: `/v3/projects/f6c43b2ef21748979caa05522db599a4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:05.015964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c43b2ef21748979caa05522db599a4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:05.017915 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:05.117328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c43b2ef21748979caa05522db599a4', 'request_id': 'req-1d20f03e-1eab-4379-b34d-cd34c7a48036'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:05.123352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c43b2ef21748979caa05522db599a4', 'request_id': 'req-1d20f03e-1eab-4379-b34d-cd34c7a48036'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:05.191233 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c43b2ef21748979caa05522db599a4', 'request_id': 'req-1d20f03e-1eab-4379-b34d-cd34c7a48036'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:05.202954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] The token cache is being invalidate because project f6c43b2ef21748979caa05522db599a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:05.203247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:05.203905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:05.205375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1262/2508] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:04 2022] DELETE /identity/v3/projects/f6c43b2ef21748979caa05522db599a4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:05.209472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae392cf-6f07-4f83-ad49-4d8a8d4dbc78 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:05.218969 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae392cf-6f07-4f83-ad49-4d8a8d4dbc78 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:05.235054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:05.274980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e78863-6130-4071-8893-e5664288a942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:05.275597 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e78863-6130-4071-8893-e5664288a942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:05.275912 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e78863-6130-4071-8893-e5664288a942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:05.276193 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e78863-6130-4071-8893-e5664288a942 admin admin] PATH_INFO: `/v3/users/f13abfbd293f435195000e1dbac9d873` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:05.277334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e78863-6130-4071-8893-e5664288a942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f13abfbd293f435195000e1dbac9d873)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:05.279316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e78863-6130-4071-8893-e5664288a942 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:05.479174 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-05e78863-6130-4071-8893-e5664288a942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f13abfbd293f435195000e1dbac9d873'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:05.551497 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-05e78863-6130-4071-8893-e5664288a942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f13abfbd293f435195000e1dbac9d873'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:05.552154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-05e78863-6130-4071-8893-e5664288a942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f13abfbd293f435195000e1dbac9d873'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:05.552778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-05e78863-6130-4071-8893-e5664288a942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f13abfbd293f435195000e1dbac9d873'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:05.560543 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-05e78863-6130-4071-8893-e5664288a942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f13abfbd293f435195000e1dbac9d873'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:05.578536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1247/2509] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:05 2022] DELETE /identity/v3/users/f13abfbd293f435195000e1dbac9d873 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:06.005138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:06.008803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:06.020945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:06.024666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20f03e-1eab-4379-b34d-cd34c7a48036 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:06.082215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:06.082846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:06.083127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:06.083401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] PATH_INFO: `/v3/projects/c9c9252d92ca497b8812d0bbe13ad027` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:06.084630 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9c9252d92ca497b8812d0bbe13ad027)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:06.086552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:06.184535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9c9252d92ca497b8812d0bbe13ad027', 'request_id': 'req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:06.190163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9c9252d92ca497b8812d0bbe13ad027', 'request_id': 'req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:06.248482 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9c9252d92ca497b8812d0bbe13ad027', 'request_id': 'req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:06.259391 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] The token cache is being invalidate because project c9c9252d92ca497b8812d0bbe13ad027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:06.259679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:06.260332 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fbdc6cbf-eed8-4a4f-8c62-15be5edbff9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:06.261804 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1263/2510] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:06 2022] DELETE /identity/v3/projects/c9c9252d92ca497b8812d0bbe13ad027 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:32:06.266664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0902ce-0e48-45fa-889a-371f24220410 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:06.266950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0902ce-0e48-45fa-889a-371f24220410 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:06.267227 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0902ce-0e48-45fa-889a-371f24220410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:06.287876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7e0902ce-0e48-45fa-889a-371f24220410 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:06.289003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7e0902ce-0e48-45fa-889a-371f24220410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:06.347477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1248/2511] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:06.356456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33713dbc-ca3f-4397-b316-74c1b773bc89 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:06.357248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33713dbc-ca3f-4397-b316-74c1b773bc89 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:06.357528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33713dbc-ca3f-4397-b316-74c1b773bc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:06.377423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-33713dbc-ca3f-4397-b316-74c1b773bc89 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:06.378623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-33713dbc-ca3f-4397-b316-74c1b773bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:06.386674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1264/2512] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:06.390654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0902ce-0e48-45fa-889a-371f24220410 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:06.400308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0902ce-0e48-45fa-889a-371f24220410 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:06.404960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab50e3-a7b6-4497-8a23-1d242b7698ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:06.405553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab50e3-a7b6-4497-8a23-1d242b7698ba admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:06.405834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab50e3-a7b6-4497-8a23-1d242b7698ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:06.406109 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab50e3-a7b6-4497-8a23-1d242b7698ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:06.406751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab50e3-a7b6-4497-8a23-1d242b7698ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:32:06.407055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab50e3-a7b6-4497-8a23-1d242b7698ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:06.408679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab50e3-a7b6-4497-8a23-1d242b7698ba admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:06.421156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1249/2513] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:32:06.425268 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33713dbc-ca3f-4397-b316-74c1b773bc89 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:06.434481 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33713dbc-ca3f-4397-b316-74c1b773bc89 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:06.440052 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf8bc-5b38-4e1b-a3f8-b4b5fd7830d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:06.440649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf8bc-5b38-4e1b-a3f8-b4b5fd7830d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:06.440929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf8bc-5b38-4e1b-a3f8-b4b5fd7830d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:06.441202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf8bc-5b38-4e1b-a3f8-b4b5fd7830d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:06.441829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aaf8bc-5b38-4e1b-a3f8-b4b5fd7830d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:06.443726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47aaf8bc-5b38-4e1b-a3f8-b4b5fd7830d1 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:06.518320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1265/2514] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:32:06 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:06.522490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab50e3-a7b6-4497-8a23-1d242b7698ba admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:06.533166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab50e3-a7b6-4497-8a23-1d242b7698ba admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:06.557673 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:06.592717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ef00a-e074-4b5e-827f-9c8efd0be726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:06.593367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ef00a-e074-4b5e-827f-9c8efd0be726 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:06.593574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ef00a-e074-4b5e-827f-9c8efd0be726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:06.593842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ef00a-e074-4b5e-827f-9c8efd0be726 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:06.594472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ef00a-e074-4b5e-827f-9c8efd0be726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:06.596366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48ef00a-e074-4b5e-827f-9c8efd0be726 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:06.735196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1250/2515] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:32:06 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:06.739276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf8bc-5b38-4e1b-a3f8-b4b5fd7830d1 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:06.749650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf8bc-5b38-4e1b-a3f8-b4b5fd7830d1 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:06.754191 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aed4c3-f2b0-44d2-87f3-fdb9a4ccf5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:06.754858 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aed4c3-f2b0-44d2-87f3-fdb9a4ccf5da admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:06.755136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aed4c3-f2b0-44d2-87f3-fdb9a4ccf5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:06.755408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aed4c3-f2b0-44d2-87f3-fdb9a4ccf5da admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:06.756072 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aed4c3-f2b0-44d2-87f3-fdb9a4ccf5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:06.757660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aed4c3-f2b0-44d2-87f3-fdb9a4ccf5da admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:06.767492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1266/2516] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:06.771287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ef00a-e074-4b5e-827f-9c8efd0be726 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:06.781792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ef00a-e074-4b5e-827f-9c8efd0be726 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:06.786375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3021171-98d7-4623-a29d-485e1389a6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:06.786994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3021171-98d7-4623-a29d-485e1389a6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:06.787275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3021171-98d7-4623-a29d-485e1389a6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:06.787548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3021171-98d7-4623-a29d-485e1389a6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:06.788212 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3021171-98d7-4623-a29d-485e1389a6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:06.789954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3021171-98d7-4623-a29d-485e1389a6d0 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:06.800486 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1251/2517] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:06.804646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aed4c3-f2b0-44d2-87f3-fdb9a4ccf5da admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:06.815062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aed4c3-f2b0-44d2-87f3-fdb9a4ccf5da admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:06.819466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71ac06-cf97-4092-acf5-2cfa46cc39fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:06.820202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71ac06-cf97-4092-acf5-2cfa46cc39fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:06.820484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71ac06-cf97-4092-acf5-2cfa46cc39fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:06.820760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71ac06-cf97-4092-acf5-2cfa46cc39fd admin admin] PATH_INFO: `/v3/projects/0b06d3aec8044897bd3487c6a1d25309/users/cf12a42a72ae423dadcc8b01882e15ee/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:06.834996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71ac06-cf97-4092-acf5-2cfa46cc39fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b06d3aec8044897bd3487c6a1d25309, user_id=cf12a42a72ae423dadcc8b01882e15ee, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:06.837749 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71ac06-cf97-4092-acf5-2cfa46cc39fd admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:06.917809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1267/2518] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:06 2022] PUT /identity/v3/projects/0b06d3aec8044897bd3487c6a1d25309/users/cf12a42a72ae423dadcc8b01882e15ee/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:06.928645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543754d-9d2f-4e87-8d7f-ea415b578758 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:06.931980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543754d-9d2f-4e87-8d7f-ea415b578758 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:06.931980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543754d-9d2f-4e87-8d7f-ea415b578758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:06.957086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-d543754d-9d2f-4e87-8d7f-ea415b578758 None None] MFA Rules not processed for user `cf12a42a72ae423dadcc8b01882e15ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:06.960703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d543754d-9d2f-4e87-8d7f-ea415b578758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:07.021594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1252/2519] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:07.026035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71ac06-cf97-4092-acf5-2cfa46cc39fd admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:07.058146 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:07.086979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71ac06-cf97-4092-acf5-2cfa46cc39fd admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:07.092168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9092153-5632-478d-b69e-82099c0e5878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:07.092775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9092153-5632-478d-b69e-82099c0e5878 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:07.093053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9092153-5632-478d-b69e-82099c0e5878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:07.093325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9092153-5632-478d-b69e-82099c0e5878 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:07.093953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9092153-5632-478d-b69e-82099c0e5878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:07.095874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9092153-5632-478d-b69e-82099c0e5878 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:07.232827 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1268/2520] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:32:07 2022] POST /identity/v3/projects => generated 387 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:07.237008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543754d-9d2f-4e87-8d7f-ea415b578758 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:07.248591 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543754d-9d2f-4e87-8d7f-ea415b578758 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:07.303796 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ba401-a15d-4427-af18-8850b637bcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:07.304399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ba401-a15d-4427-af18-8850b637bcc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:07.304678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ba401-a15d-4427-af18-8850b637bcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:07.304949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ba401-a15d-4427-af18-8850b637bcc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:07.305565 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ba401-a15d-4427-af18-8850b637bcc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:07.307463 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ba401-a15d-4427-af18-8850b637bcc1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:07.401697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1253/2521] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:32:07 2022] POST /identity/v3/users => generated 300 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:07.406501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9092153-5632-478d-b69e-82099c0e5878 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:07.416744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9092153-5632-478d-b69e-82099c0e5878 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:07.421434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dc5c3-0958-44fc-8ae5-6f97990b51b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:07.422026 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dc5c3-0958-44fc-8ae5-6f97990b51b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:07.422306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dc5c3-0958-44fc-8ae5-6f97990b51b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:07.422581 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dc5c3-0958-44fc-8ae5-6f97990b51b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:07.423242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90dc5c3-0958-44fc-8ae5-6f97990b51b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:07.424906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90dc5c3-0958-44fc-8ae5-6f97990b51b3 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:07.434756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1269/2522] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:07.439567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ba401-a15d-4427-af18-8850b637bcc1 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:07.449350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ba401-a15d-4427-af18-8850b637bcc1 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:07.454419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5081ad-6312-4e3e-91cd-441656ed4817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:07.455052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5081ad-6312-4e3e-91cd-441656ed4817 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:07.455328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5081ad-6312-4e3e-91cd-441656ed4817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:07.455597 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5081ad-6312-4e3e-91cd-441656ed4817 admin admin] PATH_INFO: `/v3/projects/65fbb7c78f2d4eb1bf3b6efd8d3bdd18/users/d1b3534a577547e9adfe0e148514320d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:07.469844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5081ad-6312-4e3e-91cd-441656ed4817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fbb7c78f2d4eb1bf3b6efd8d3bdd18, user_id=d1b3534a577547e9adfe0e148514320d, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:07.472475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5081ad-6312-4e3e-91cd-441656ed4817 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:07.552488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:07.556921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1254/2523] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:07 2022] PUT /identity/v3/projects/65fbb7c78f2d4eb1bf3b6efd8d3bdd18/users/d1b3534a577547e9adfe0e148514320d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:07.561400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dc5c3-0958-44fc-8ae5-6f97990b51b3 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:07.571707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dc5c3-0958-44fc-8ae5-6f97990b51b3 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:07.626411 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644980ac-4fa6-4d28-a930-69b073957567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:07.627006 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644980ac-4fa6-4d28-a930-69b073957567 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:07.627285 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644980ac-4fa6-4d28-a930-69b073957567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:07.627558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644980ac-4fa6-4d28-a930-69b073957567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:07.628216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644980ac-4fa6-4d28-a930-69b073957567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:07.629804 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644980ac-4fa6-4d28-a930-69b073957567 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:07.639891 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1270/2524] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:32:07.643960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5081ad-6312-4e3e-91cd-441656ed4817 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:07.654724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5081ad-6312-4e3e-91cd-441656ed4817 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:07.659424 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7acf6-e353-40f8-a5f5-3c3033a4f0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:07.660351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7acf6-e353-40f8-a5f5-3c3033a4f0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:07.661003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7acf6-e353-40f8-a5f5-3c3033a4f0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:07.661334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c7acf6-e353-40f8-a5f5-3c3033a4f0e9 admin admin] PATH_INFO: `/v3/projects/65fbb7c78f2d4eb1bf3b6efd8d3bdd18/users/d1b3534a577547e9adfe0e148514320d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:07.663413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7acf6-e353-40f8-a5f5-3c3033a4f0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65fbb7c78f2d4eb1bf3b6efd8d3bdd18, user_id=d1b3534a577547e9adfe0e148514320d, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:07.666143 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c7acf6-e353-40f8-a5f5-3c3033a4f0e9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:07.676699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1255/2525] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:07 2022] PUT /identity/v3/projects/65fbb7c78f2d4eb1bf3b6efd8d3bdd18/users/d1b3534a577547e9adfe0e148514320d/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:07.686776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946783f-9ff1-4d2b-aa53-39ba621f936d None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:07.687062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946783f-9ff1-4d2b-aa53-39ba621f936d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:07.687432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946783f-9ff1-4d2b-aa53-39ba621f936d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:07.715038 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-a946783f-9ff1-4d2b-aa53-39ba621f936d None None] MFA Rules not processed for user `d1b3534a577547e9adfe0e148514320d`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:07.716841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a946783f-9ff1-4d2b-aa53-39ba621f936d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:07.784876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1271/2526] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:07.796832 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7acf6-e353-40f8-a5f5-3c3033a4f0e9 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:07.798494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-76c7acf6-e353-40f8-a5f5-3c3033a4f0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:07.854820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c7acf6-e353-40f8-a5f5-3c3033a4f0e9 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:07.858417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc807a-4572-40b7-8166-d4ec0b9c6d6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:07.858998 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc807a-4572-40b7-8166-d4ec0b9c6d6a service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:07.859271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc807a-4572-40b7-8166-d4ec0b9c6d6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:07.859539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc807a-4572-40b7-8166-d4ec0b9c6d6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:07.868277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc807a-4572-40b7-8166-d4ec0b9c6d6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:07.869692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc807a-4572-40b7-8166-d4ec0b9c6d6a service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:07.878881 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1256/2527] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:07 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:32:07.943006 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946783f-9ff1-4d2b-aa53-39ba621f936d None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:07.973422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:08.007914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946783f-9ff1-4d2b-aa53-39ba621f936d None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:08.012539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:08.013195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:08.013477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:08.013744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] PATH_INFO: `/v3/users/cf12a42a72ae423dadcc8b01882e15ee` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:08.014862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf12a42a72ae423dadcc8b01882e15ee)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:08.016842 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:08.181698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf12a42a72ae423dadcc8b01882e15ee'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:08.238009 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf12a42a72ae423dadcc8b01882e15ee'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:08.238666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf12a42a72ae423dadcc8b01882e15ee'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:08.239941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf12a42a72ae423dadcc8b01882e15ee'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:08.247523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf12a42a72ae423dadcc8b01882e15ee'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:08.264942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1272/2528] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:07 2022] DELETE /identity/v3/users/cf12a42a72ae423dadcc8b01882e15ee => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:08.269392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9249f-029e-4dd9-b235-e2e1245ea0fa None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:08.270827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9249f-029e-4dd9-b235-e2e1245ea0fa None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:08.270827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9249f-029e-4dd9-b235-e2e1245ea0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:08.289352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-f3f9249f-029e-4dd9-b235-e2e1245ea0fa None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:08.290641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f3f9249f-029e-4dd9-b235-e2e1245ea0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:08.348516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1257/2529] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:08.358164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:08.359557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:08.416498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd8168-050a-45cd-854a-497a24184072 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:08.420168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77de68-ce7d-492d-9202-49134a64f268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:08.420797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77de68-ce7d-492d-9202-49134a64f268 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:08.421097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77de68-ce7d-492d-9202-49134a64f268 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:08.421372 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f77de68-ce7d-492d-9202-49134a64f268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:08.430104 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77de68-ce7d-492d-9202-49134a64f268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:08.431522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f77de68-ce7d-492d-9202-49134a64f268 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:08.441626 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1273/2530] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:08.820749 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9249f-029e-4dd9-b235-e2e1245ea0fa None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:08.822222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f3f9249f-029e-4dd9-b235-e2e1245ea0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:08.836744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9249f-029e-4dd9-b235-e2e1245ea0fa None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:08.842079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:08.842718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:08.842997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:08.843273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] PATH_INFO: `/v3/projects/0b06d3aec8044897bd3487c6a1d25309` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:08.844431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b06d3aec8044897bd3487c6a1d25309)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:08.846467 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:08.946622 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b06d3aec8044897bd3487c6a1d25309', 'request_id': 'req-8d528707-c546-41d7-8e1c-338a0eb17d1d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:09.014656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b06d3aec8044897bd3487c6a1d25309', 'request_id': 'req-8d528707-c546-41d7-8e1c-338a0eb17d1d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:09.021468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b06d3aec8044897bd3487c6a1d25309', 'request_id': 'req-8d528707-c546-41d7-8e1c-338a0eb17d1d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:09.032472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] The token cache is being invalidate because project 0b06d3aec8044897bd3487c6a1d25309 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:09.036449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:09.036449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:09.036449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1258/2531] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:08 2022] DELETE /identity/v3/projects/0b06d3aec8044897bd3487c6a1d25309 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:09.038863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77de68-ce7d-492d-9202-49134a64f268 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:09.049108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f77de68-ce7d-492d-9202-49134a64f268 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:09.093643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:09.104973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:09.105726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:09.105998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:09.106267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] PATH_INFO: `/v3/users/d1b3534a577547e9adfe0e148514320d` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:09.108031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b3534a577547e9adfe0e148514320d)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:09.109928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:09.295602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b3534a577547e9adfe0e148514320d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:09.345242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b3534a577547e9adfe0e148514320d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:09.345881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b3534a577547e9adfe0e148514320d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:09.346526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b3534a577547e9adfe0e148514320d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:09.353675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b3534a577547e9adfe0e148514320d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:09.370986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1274/2532] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:09 2022] DELETE /identity/v3/users/d1b3534a577547e9adfe0e148514320d => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:09.753506 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:09.754938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:09.771509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d528707-c546-41d7-8e1c-338a0eb17d1d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:09.786886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:09.831172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:09.831804 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:09.832033 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:09.832307 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] PATH_INFO: `/v3/projects/65fbb7c78f2d4eb1bf3b6efd8d3bdd18` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:09.833428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65fbb7c78f2d4eb1bf3b6efd8d3bdd18)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:09.835336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:09.876191 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:09.877681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:09.937496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e131c4-8270-4fc1-b585-954d1f7c1405 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:09.941523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44aa906-010a-44c8-9311-185ecee02d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:09.945152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aa906-010a-44c8-9311-185ecee02d15 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:09.945152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aa906-010a-44c8-9311-185ecee02d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:09.945152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aa906-010a-44c8-9311-185ecee02d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:09.945152 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d44aa906-010a-44c8-9311-185ecee02d15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:09.955033 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65fbb7c78f2d4eb1bf3b6efd8d3bdd18', 'request_id': 'req-937201c1-f9e3-4ef3-820e-7ce3c6057266'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:10.007814 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44aa906-010a-44c8-9311-185ecee02d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:10.009262 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44aa906-010a-44c8-9311-185ecee02d15 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:10.019011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1275/2533] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:32:10.025816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65fbb7c78f2d4eb1bf3b6efd8d3bdd18', 'request_id': 'req-937201c1-f9e3-4ef3-820e-7ce3c6057266'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:10.033855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65fbb7c78f2d4eb1bf3b6efd8d3bdd18', 'request_id': 'req-937201c1-f9e3-4ef3-820e-7ce3c6057266'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:10.045142 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] The token cache is being invalidate because project 65fbb7c78f2d4eb1bf3b6efd8d3bdd18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:10.046659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:10.046659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-937201c1-f9e3-4ef3-820e-7ce3c6057266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:10.047740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1259/2534] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:09 2022] DELETE /identity/v3/projects/65fbb7c78f2d4eb1bf3b6efd8d3bdd18 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:10.053084 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cd522-ddeb-4c1a-b039-e71558fd9183 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:10.053084 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cd522-ddeb-4c1a-b039-e71558fd9183 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:10.053320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cd522-ddeb-4c1a-b039-e71558fd9183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:10.073554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-4d0cd522-ddeb-4c1a-b039-e71558fd9183 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:10.074757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4d0cd522-ddeb-4c1a-b039-e71558fd9183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:10.094277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:10.134724 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1276/2535] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:10.144081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d55d9-ba3c-4558-9afa-dca3ab1c3901 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:10.144458 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d55d9-ba3c-4558-9afa-dca3ab1c3901 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:10.144757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d55d9-ba3c-4558-9afa-dca3ab1c3901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:10.164911 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-f12d55d9-ba3c-4558-9afa-dca3ab1c3901 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:10.167343 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f12d55d9-ba3c-4558-9afa-dca3ab1c3901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:10.173988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1260/2536] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:10.178267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cd522-ddeb-4c1a-b039-e71558fd9183 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:10.188704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cd522-ddeb-4c1a-b039-e71558fd9183 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:10.193384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5506395-ac81-47cd-aab9-6900e00ff07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:10.193981 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5506395-ac81-47cd-aab9-6900e00ff07e admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:10.194258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5506395-ac81-47cd-aab9-6900e00ff07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:10.194528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5506395-ac81-47cd-aab9-6900e00ff07e admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:10.195166 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5506395-ac81-47cd-aab9-6900e00ff07e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:32:10.195470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5506395-ac81-47cd-aab9-6900e00ff07e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:10.197101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5506395-ac81-47cd-aab9-6900e00ff07e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:10.209563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1277/2537] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:32:10.213356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d55d9-ba3c-4558-9afa-dca3ab1c3901 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:10.222789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d55d9-ba3c-4558-9afa-dca3ab1c3901 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:10.227438 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62c688-b90b-46bd-9ad4-9d0b66b38e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:10.228058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62c688-b90b-46bd-9ad4-9d0b66b38e06 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:10.228337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62c688-b90b-46bd-9ad4-9d0b66b38e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:10.228608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62c688-b90b-46bd-9ad4-9d0b66b38e06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:10.229259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62c688-b90b-46bd-9ad4-9d0b66b38e06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:10.231198 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62c688-b90b-46bd-9ad4-9d0b66b38e06 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:10.329658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1261/2538] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:32:10 2022] POST /identity/v3/projects => generated 391 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:10.333850 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5506395-ac81-47cd-aab9-6900e00ff07e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:10.344639 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5506395-ac81-47cd-aab9-6900e00ff07e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:10.402320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035650f5-e6fb-4e71-b7f3-6c651d170c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:10.402918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035650f5-e6fb-4e71-b7f3-6c651d170c8b admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:10.403195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035650f5-e6fb-4e71-b7f3-6c651d170c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:10.403467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035650f5-e6fb-4e71-b7f3-6c651d170c8b admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:10.404119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035650f5-e6fb-4e71-b7f3-6c651d170c8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:10.406538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035650f5-e6fb-4e71-b7f3-6c651d170c8b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:10.523203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1278/2539] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:32:10 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:10.527340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62c688-b90b-46bd-9ad4-9d0b66b38e06 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:10.538124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62c688-b90b-46bd-9ad4-9d0b66b38e06 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:10.542705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4bc92-c864-43fd-9760-c22907427c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:10.543300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4bc92-c864-43fd-9760-c22907427c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:10.543581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4bc92-c864-43fd-9760-c22907427c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:10.543885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4bc92-c864-43fd-9760-c22907427c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:10.544522 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4bc92-c864-43fd-9760-c22907427c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:10.546140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4bc92-c864-43fd-9760-c22907427c88 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:10.556428 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1262/2540] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:10.560369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035650f5-e6fb-4e71-b7f3-6c651d170c8b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:10.571051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035650f5-e6fb-4e71-b7f3-6c651d170c8b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:10.575705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c2ae8-1aa2-43a2-9eea-1163d5954b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:10.576838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c2ae8-1aa2-43a2-9eea-1163d5954b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:10.577121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c2ae8-1aa2-43a2-9eea-1163d5954b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:10.577395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c2ae8-1aa2-43a2-9eea-1163d5954b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:10.578032 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c2ae8-1aa2-43a2-9eea-1163d5954b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:10.579649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8c2ae8-1aa2-43a2-9eea-1163d5954b89 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:10.590264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1279/2541] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:10.596621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4bc92-c864-43fd-9760-c22907427c88 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:10.604896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4bc92-c864-43fd-9760-c22907427c88 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:10.610009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15f6c3-6ae6-4284-b0ff-f6dba2e8850e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:10.610650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15f6c3-6ae6-4284-b0ff-f6dba2e8850e admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:10.610938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15f6c3-6ae6-4284-b0ff-f6dba2e8850e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:10.611209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a15f6c3-6ae6-4284-b0ff-f6dba2e8850e admin admin] PATH_INFO: `/v3/projects/090ad2d919f244d4bdf8d4d78c5fb971/users/6c4f7bc4801348339886a2c3352e2292/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:10.626228 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15f6c3-6ae6-4284-b0ff-f6dba2e8850e admin admin] RBAC: Authorizing `identity:create_grant(project_id=090ad2d919f244d4bdf8d4d78c5fb971, user_id=6c4f7bc4801348339886a2c3352e2292, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:10.628909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a15f6c3-6ae6-4284-b0ff-f6dba2e8850e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:10.716639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1263/2542] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:10 2022] PUT /identity/v3/projects/090ad2d919f244d4bdf8d4d78c5fb971/users/6c4f7bc4801348339886a2c3352e2292/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:10.727576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac32e7-eb58-4cba-971b-88689eab32da None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:10.727888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac32e7-eb58-4cba-971b-88689eab32da None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:10.728170 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac32e7-eb58-4cba-971b-88689eab32da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:10.757253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-19ac32e7-eb58-4cba-971b-88689eab32da None None] MFA Rules not processed for user `6c4f7bc4801348339886a2c3352e2292`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:10.758491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-19ac32e7-eb58-4cba-971b-88689eab32da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:10.824100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1280/2543] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:10.828567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15f6c3-6ae6-4284-b0ff-f6dba2e8850e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:10.871291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:10.890009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a15f6c3-6ae6-4284-b0ff-f6dba2e8850e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:10.894694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45915fd1-f9e9-4bb2-bd07-0391b0b64f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:10.895288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45915fd1-f9e9-4bb2-bd07-0391b0b64f57 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:10.895566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45915fd1-f9e9-4bb2-bd07-0391b0b64f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:10.895870 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45915fd1-f9e9-4bb2-bd07-0391b0b64f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:10.896501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45915fd1-f9e9-4bb2-bd07-0391b0b64f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:10.898482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45915fd1-f9e9-4bb2-bd07-0391b0b64f57 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:10.997476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1264/2544] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:32:10 2022] POST /identity/v3/projects => generated 391 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:32:11.002137 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac32e7-eb58-4cba-971b-88689eab32da None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:11.012905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ac32e7-eb58-4cba-971b-88689eab32da None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:11.025981 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:11.069496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a3cf0-fece-4118-9de3-ea1e75b9ab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:11.070103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a3cf0-fece-4118-9de3-ea1e75b9ab20 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:11.070436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a3cf0-fece-4118-9de3-ea1e75b9ab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:11.070712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a3cf0-fece-4118-9de3-ea1e75b9ab20 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:11.071340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a3cf0-fece-4118-9de3-ea1e75b9ab20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:11.073301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a3cf0-fece-4118-9de3-ea1e75b9ab20 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:11.161845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1281/2545] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:32:11 2022] POST /identity/v3/users => generated 302 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:32:11.165901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45915fd1-f9e9-4bb2-bd07-0391b0b64f57 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:11.176087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45915fd1-f9e9-4bb2-bd07-0391b0b64f57 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:11.180714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503abd9e-29f8-43c8-8596-4f6f05c9b034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:11.185532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503abd9e-29f8-43c8-8596-4f6f05c9b034 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:11.185532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503abd9e-29f8-43c8-8596-4f6f05c9b034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:11.185532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503abd9e-29f8-43c8-8596-4f6f05c9b034 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:11.185532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503abd9e-29f8-43c8-8596-4f6f05c9b034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:11.185532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503abd9e-29f8-43c8-8596-4f6f05c9b034 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:11.194671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1265/2546] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:11.198544 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a3cf0-fece-4118-9de3-ea1e75b9ab20 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:11.209297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a3cf0-fece-4118-9de3-ea1e75b9ab20 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:11.213866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db453f5b-4562-40a7-84d1-1e885d95b3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:11.214504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db453f5b-4562-40a7-84d1-1e885d95b3d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:11.214780 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db453f5b-4562-40a7-84d1-1e885d95b3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:11.215054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db453f5b-4562-40a7-84d1-1e885d95b3d9 admin admin] PATH_INFO: `/v3/projects/ee7925a086084c109ce926e1f1edaf39/users/7cb370f4136742539cf2804b20031a35/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:11.229697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db453f5b-4562-40a7-84d1-1e885d95b3d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7925a086084c109ce926e1f1edaf39, user_id=7cb370f4136742539cf2804b20031a35, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:11.232478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db453f5b-4562-40a7-84d1-1e885d95b3d9 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:11.339676 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1282/2547] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:11 2022] PUT /identity/v3/projects/ee7925a086084c109ce926e1f1edaf39/users/7cb370f4136742539cf2804b20031a35/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:11.344382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503abd9e-29f8-43c8-8596-4f6f05c9b034 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:11.354386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503abd9e-29f8-43c8-8596-4f6f05c9b034 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:11.409579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdcadc-fe73-40f4-8939-337d45bb0ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:11.410184 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdcadc-fe73-40f4-8939-337d45bb0ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:11.410468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdcadc-fe73-40f4-8939-337d45bb0ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:11.410744 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdcadc-fe73-40f4-8939-337d45bb0ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:11.411385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdcadc-fe73-40f4-8939-337d45bb0ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:11.413106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdcadc-fe73-40f4-8939-337d45bb0ac9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:11.422823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1266/2548] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:11.426920 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db453f5b-4562-40a7-84d1-1e885d95b3d9 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:11.437114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db453f5b-4562-40a7-84d1-1e885d95b3d9 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:11.441956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64a171-3188-43a1-8240-a4b555315d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:11.442600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64a171-3188-43a1-8240-a4b555315d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:11.442880 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64a171-3188-43a1-8240-a4b555315d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:11.443155 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64a171-3188-43a1-8240-a4b555315d46 admin admin] PATH_INFO: `/v3/projects/ee7925a086084c109ce926e1f1edaf39/users/7cb370f4136742539cf2804b20031a35/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:11.445084 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64a171-3188-43a1-8240-a4b555315d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7925a086084c109ce926e1f1edaf39, user_id=7cb370f4136742539cf2804b20031a35, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:11.447705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64a171-3188-43a1-8240-a4b555315d46 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:11.458455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1283/2549] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:11 2022] PUT /identity/v3/projects/ee7925a086084c109ce926e1f1edaf39/users/7cb370f4136742539cf2804b20031a35/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:11.473762 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b389dd-819f-4042-aa91-48ead16ed4d8 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:11.474048 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b389dd-819f-4042-aa91-48ead16ed4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:11.474323 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b389dd-819f-4042-aa91-48ead16ed4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:11.504461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-90b389dd-819f-4042-aa91-48ead16ed4d8 None None] MFA Rules not processed for user `7cb370f4136742539cf2804b20031a35`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:11.505702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-90b389dd-819f-4042-aa91-48ead16ed4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:11.573554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1267/2550] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:11 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:11.585404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64a171-3188-43a1-8240-a4b555315d46 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:11.587091 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5f64a171-3188-43a1-8240-a4b555315d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:11.644822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64a171-3188-43a1-8240-a4b555315d46 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:11.648528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeea724-17b8-476e-92bf-3f4dd2ced7e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:11.649133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeea724-17b8-476e-92bf-3f4dd2ced7e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:11.649414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeea724-17b8-476e-92bf-3f4dd2ced7e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:11.649687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeea724-17b8-476e-92bf-3f4dd2ced7e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:11.659091 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeea724-17b8-476e-92bf-3f4dd2ced7e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:11.660553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeea724-17b8-476e-92bf-3f4dd2ced7e9 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:11.670593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1284/2551] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:11 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:11.715639 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b389dd-819f-4042-aa91-48ead16ed4d8 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:11.717637 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-90b389dd-819f-4042-aa91-48ead16ed4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:11.729817 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b389dd-819f-4042-aa91-48ead16ed4d8 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:11.734050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941976aa-609d-45ee-876a-16206f1f470e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:11.734644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941976aa-609d-45ee-876a-16206f1f470e service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:11.734923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941976aa-609d-45ee-876a-16206f1f470e service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:11.735196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941976aa-609d-45ee-876a-16206f1f470e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:11.767197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:11.794151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941976aa-609d-45ee-876a-16206f1f470e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:11.796259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941976aa-609d-45ee-876a-16206f1f470e service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:11.805702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1268/2552] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:32:21.665723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=62,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:21.905723 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=12 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:22.728684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54432-b08d-4ca6-8c4e-dbcec959c7b0 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:22.729193 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54432-b08d-4ca6-8c4e-dbcec959c7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:22.729304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d54432-b08d-4ca6-8c4e-dbcec959c7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:22.748078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e2d54432-b08d-4ca6-8c4e-dbcec959c7b0 None None] MFA Rules not processed for user `be10df0a4d1d42208c46615d6ddd970c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:22.749260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e2d54432-b08d-4ca6-8c4e-dbcec959c7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:22.803250 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1285/2553] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:22.841567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941976aa-609d-45ee-876a-16206f1f470e service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:22.850763 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941976aa-609d-45ee-876a-16206f1f470e service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:22.854658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e1bb9-c863-4a95-b53c-1b374e39277b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:22.855243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e1bb9-c863-4a95-b53c-1b374e39277b service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:22.855525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e1bb9-c863-4a95-b53c-1b374e39277b service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:22.855893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e1bb9-c863-4a95-b53c-1b374e39277b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:22.857765 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ee8e1bb9-c863-4a95-b53c-1b374e39277b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:22.929821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e1bb9-c863-4a95-b53c-1b374e39277b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:22.931258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e1bb9-c863-4a95-b53c-1b374e39277b service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:22.943292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1269/2554] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:23.004457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d54432-b08d-4ca6-8c4e-dbcec959c7b0 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:23.005892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e2d54432-b08d-4ca6-8c4e-dbcec959c7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:23.090223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d54432-b08d-4ca6-8c4e-dbcec959c7b0 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:23.094355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa5878-a220-4d75-9a9c-3278c26bda79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:23.096411 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa5878-a220-4d75-9a9c-3278c26bda79 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:23.096973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa5878-a220-4d75-9a9c-3278c26bda79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:23.097412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa5878-a220-4d75-9a9c-3278c26bda79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:23.099919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f8aa5878-a220-4d75-9a9c-3278c26bda79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:23.182470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa5878-a220-4d75-9a9c-3278c26bda79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:23.184323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa5878-a220-4d75-9a9c-3278c26bda79 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:23.194695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1286/2555] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:23.210655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e1bb9-c863-4a95-b53c-1b374e39277b service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:23.219471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e1bb9-c863-4a95-b53c-1b374e39277b service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:23.225649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80f488-c685-4814-ab4c-2665a23f931d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:23.225649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80f488-c685-4814-ab4c-2665a23f931d service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:23.225649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80f488-c685-4814-ab4c-2665a23f931d service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:23.226106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80f488-c685-4814-ab4c-2665a23f931d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:23.227658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6b80f488-c685-4814-ab4c-2665a23f931d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:23.244638 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80f488-c685-4814-ab4c-2665a23f931d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:23.246092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80f488-c685-4814-ab4c-2665a23f931d service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:23.263834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1270/2556] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:23.685154 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa5878-a220-4d75-9a9c-3278c26bda79 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:23.756316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa5878-a220-4d75-9a9c-3278c26bda79 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:23.761026 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:23.761663 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:23.761978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:23.762252 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] PATH_INFO: `/v3/users/6c4f7bc4801348339886a2c3352e2292` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:23.763389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4f7bc4801348339886a2c3352e2292)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:23.765363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:23.767729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:23.931851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4f7bc4801348339886a2c3352e2292'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:23.991983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4f7bc4801348339886a2c3352e2292'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:23.992612 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4f7bc4801348339886a2c3352e2292'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:23.993424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4f7bc4801348339886a2c3352e2292'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:24.002175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4f7bc4801348339886a2c3352e2292'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:24.026343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1287/2557] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:23 2022] DELETE /identity/v3/users/6c4f7bc4801348339886a2c3352e2292 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:24.030973 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e87782-4299-49bc-9db7-40d799e4017d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:24.040436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e87782-4299-49bc-9db7-40d799e4017d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:24.040436 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e87782-4299-49bc-9db7-40d799e4017d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:24.066004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-d0e87782-4299-49bc-9db7-40d799e4017d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:24.068395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d0e87782-4299-49bc-9db7-40d799e4017d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:24.148848 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1271/2558] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:24.167405 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:24.170499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:24.302243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d6fbc2-0d9f-43bb-bee9-c629d05c95cc admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:24.306043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d3847-8f69-414d-9084-103ad7f8f081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:24.306790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d3847-8f69-414d-9084-103ad7f8f081 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:24.307074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d3847-8f69-414d-9084-103ad7f8f081 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:24.307472 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d3847-8f69-414d-9084-103ad7f8f081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:24.317068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d3847-8f69-414d-9084-103ad7f8f081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:24.318640 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00d3847-8f69-414d-9084-103ad7f8f081 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:24.328314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1288/2559] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:24.793246 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e87782-4299-49bc-9db7-40d799e4017d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:24.795089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d0e87782-4299-49bc-9db7-40d799e4017d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:24.809836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e87782-4299-49bc-9db7-40d799e4017d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:24.815495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:24.816311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:24.816631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:24.816981 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] PATH_INFO: `/v3/projects/090ad2d919f244d4bdf8d4d78c5fb971` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:24.818278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=090ad2d919f244d4bdf8d4d78c5fb971)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:24.820601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:24.948050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '090ad2d919f244d4bdf8d4d78c5fb971', 'request_id': 'req-f6bbdbcf-b36f-423a-a98f-af7686453f82'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:25.020630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '090ad2d919f244d4bdf8d4d78c5fb971', 'request_id': 'req-f6bbdbcf-b36f-423a-a98f-af7686453f82'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:25.028044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '090ad2d919f244d4bdf8d4d78c5fb971', 'request_id': 'req-f6bbdbcf-b36f-423a-a98f-af7686453f82'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:25.033126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:25.039805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] The token cache is being invalidate because project 090ad2d919f244d4bdf8d4d78c5fb971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:25.040166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:25.040905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:25.042660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1272/2560] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:24 2022] DELETE /identity/v3/projects/090ad2d919f244d4bdf8d4d78c5fb971 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:25.046752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d3847-8f69-414d-9084-103ad7f8f081 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:25.057043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d3847-8f69-414d-9084-103ad7f8f081 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:25.115803 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a549ec3-feff-424c-b0de-5c8708cbd675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:25.116543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a549ec3-feff-424c-b0de-5c8708cbd675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:25.116825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a549ec3-feff-424c-b0de-5c8708cbd675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:25.117316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a549ec3-feff-424c-b0de-5c8708cbd675 admin admin] PATH_INFO: `/v3/users/7cb370f4136742539cf2804b20031a35` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:25.118232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a549ec3-feff-424c-b0de-5c8708cbd675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb370f4136742539cf2804b20031a35)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:25.120370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a549ec3-feff-424c-b0de-5c8708cbd675 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:25.150463 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:25.308999 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-4a549ec3-feff-424c-b0de-5c8708cbd675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb370f4136742539cf2804b20031a35'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:25.392258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-4a549ec3-feff-424c-b0de-5c8708cbd675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb370f4136742539cf2804b20031a35'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:25.392883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-4a549ec3-feff-424c-b0de-5c8708cbd675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb370f4136742539cf2804b20031a35'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:25.393486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-4a549ec3-feff-424c-b0de-5c8708cbd675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb370f4136742539cf2804b20031a35'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:25.400704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-4a549ec3-feff-424c-b0de-5c8708cbd675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb370f4136742539cf2804b20031a35'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:25.417887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1289/2561] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:25 2022] DELETE /identity/v3/users/7cb370f4136742539cf2804b20031a35 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:32:25.807504 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:25.809203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:25.831086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bbdbcf-b36f-423a-a98f-af7686453f82 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:25.902018 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:25.903136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:25.903136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:25.903324 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] PATH_INFO: `/v3/projects/ee7925a086084c109ce926e1f1edaf39` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:25.904460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7925a086084c109ce926e1f1edaf39)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:25.906382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:26.046805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7925a086084c109ce926e1f1edaf39', 'request_id': 'req-5a120147-60c2-4ca2-a8c4-1aad631d9a13'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:26.134691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7925a086084c109ce926e1f1edaf39', 'request_id': 'req-5a120147-60c2-4ca2-a8c4-1aad631d9a13'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:26.142115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7925a086084c109ce926e1f1edaf39', 'request_id': 'req-5a120147-60c2-4ca2-a8c4-1aad631d9a13'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:26.152596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] The token cache is being invalidate because project ee7925a086084c109ce926e1f1edaf39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:26.152889 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:26.153518 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-5a120147-60c2-4ca2-a8c4-1aad631d9a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:26.154991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1273/2562] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:25 2022] DELETE /identity/v3/projects/ee7925a086084c109ce926e1f1edaf39 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:26.160679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeb912-0b9a-4211-b48c-295c604f66c7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:26.161161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeb912-0b9a-4211-b48c-295c604f66c7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:26.161271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeb912-0b9a-4211-b48c-295c604f66c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:26.180776 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-25eeb912-0b9a-4211-b48c-295c604f66c7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:26.182005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-25eeb912-0b9a-4211-b48c-295c604f66c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:26.247782 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1290/2563] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:32:26.256189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93509b-440e-4d8a-ac19-667637b30801 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:26.256502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93509b-440e-4d8a-ac19-667637b30801 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:26.256806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93509b-440e-4d8a-ac19-667637b30801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:26.281321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-eb93509b-440e-4d8a-ac19-667637b30801 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:26.282580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-eb93509b-440e-4d8a-ac19-667637b30801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:26.290653 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1274/2564] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:26.294831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eeb912-0b9a-4211-b48c-295c604f66c7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:26.305171 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eeb912-0b9a-4211-b48c-295c604f66c7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:26.309666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b582e-2965-4a27-a314-49eff9c8979b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:26.310261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b582e-2965-4a27-a314-49eff9c8979b admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:26.310539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b582e-2965-4a27-a314-49eff9c8979b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:26.310817 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b582e-2965-4a27-a314-49eff9c8979b admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:26.311459 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b582e-2965-4a27-a314-49eff9c8979b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:32:26.311781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b582e-2965-4a27-a314-49eff9c8979b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:26.313462 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b582e-2965-4a27-a314-49eff9c8979b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:26.326455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1291/2565] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:32:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:32:26.330537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93509b-440e-4d8a-ac19-667637b30801 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:26.339896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93509b-440e-4d8a-ac19-667637b30801 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:26.344646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f238a1-d389-4266-bed2-4c65a83e13ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:26.345243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f238a1-d389-4266-bed2-4c65a83e13ae admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:26.345529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f238a1-d389-4266-bed2-4c65a83e13ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:26.345805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f238a1-d389-4266-bed2-4c65a83e13ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:26.346437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f238a1-d389-4266-bed2-4c65a83e13ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:26.348414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f238a1-d389-4266-bed2-4c65a83e13ae admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:26.490449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1275/2566] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:32:26 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:26.494659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b582e-2965-4a27-a314-49eff9c8979b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:26.505197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b582e-2965-4a27-a314-49eff9c8979b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:26.561561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1e876-a0ec-4fe9-8896-77edd2a1078b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:26.562174 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1e876-a0ec-4fe9-8896-77edd2a1078b admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:26.562461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1e876-a0ec-4fe9-8896-77edd2a1078b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:26.562735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1e876-a0ec-4fe9-8896-77edd2a1078b admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:26.563378 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1e876-a0ec-4fe9-8896-77edd2a1078b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:26.565603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1e876-a0ec-4fe9-8896-77edd2a1078b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:26.654887 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1292/2567] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:32:26 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:26.659659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f238a1-d389-4266-bed2-4c65a83e13ae admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:26.670034 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f238a1-d389-4266-bed2-4c65a83e13ae admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:26.675299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8fb12-894c-4ed7-a14b-e40ddb9a9de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:26.675930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8fb12-894c-4ed7-a14b-e40ddb9a9de7 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:26.676342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8fb12-894c-4ed7-a14b-e40ddb9a9de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:26.676624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef8fb12-894c-4ed7-a14b-e40ddb9a9de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:26.677273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8fb12-894c-4ed7-a14b-e40ddb9a9de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:26.678876 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef8fb12-894c-4ed7-a14b-e40ddb9a9de7 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:26.690167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1276/2568] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:26.694188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1e876-a0ec-4fe9-8896-77edd2a1078b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:26.704394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1e876-a0ec-4fe9-8896-77edd2a1078b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:26.709212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e767164-07dd-4ae4-adb0-1e53ab18f238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:26.714931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e767164-07dd-4ae4-adb0-1e53ab18f238 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:26.714931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e767164-07dd-4ae4-adb0-1e53ab18f238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:26.714931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e767164-07dd-4ae4-adb0-1e53ab18f238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:26.714931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e767164-07dd-4ae4-adb0-1e53ab18f238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:26.714931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e767164-07dd-4ae4-adb0-1e53ab18f238 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:26.723633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1293/2569] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:26.727846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8fb12-894c-4ed7-a14b-e40ddb9a9de7 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:26.738951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef8fb12-894c-4ed7-a14b-e40ddb9a9de7 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:26.743573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd8e7e-88be-44d7-a0c7-7f40781ffdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:26.744302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd8e7e-88be-44d7-a0c7-7f40781ffdf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:26.744580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd8e7e-88be-44d7-a0c7-7f40781ffdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:26.744856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd8e7e-88be-44d7-a0c7-7f40781ffdf6 admin admin] PATH_INFO: `/v3/projects/c8163ad6b5274157bcc26bdc5c18f8be/users/b4af8bf91e454377a69ecbd26a2999aa/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:26.760590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd8e7e-88be-44d7-a0c7-7f40781ffdf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8163ad6b5274157bcc26bdc5c18f8be, user_id=b4af8bf91e454377a69ecbd26a2999aa, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:26.763287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd8e7e-88be-44d7-a0c7-7f40781ffdf6 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:26.827146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1277/2570] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:26 2022] PUT /identity/v3/projects/c8163ad6b5274157bcc26bdc5c18f8be/users/b4af8bf91e454377a69ecbd26a2999aa/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:26.838032 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f7ef6-fc58-456a-bbff-3713bbb1b0ba None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:26.838521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f7ef6-fc58-456a-bbff-3713bbb1b0ba None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:26.838797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f7ef6-fc58-456a-bbff-3713bbb1b0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:26.853530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:26.872879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-377f7ef6-fc58-456a-bbff-3713bbb1b0ba None None] MFA Rules not processed for user `b4af8bf91e454377a69ecbd26a2999aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:26.874202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-377f7ef6-fc58-456a-bbff-3713bbb1b0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:26.953229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1294/2571] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:26.958213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd8e7e-88be-44d7-a0c7-7f40781ffdf6 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:26.973004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:27.029300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd8e7e-88be-44d7-a0c7-7f40781ffdf6 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:27.034989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e5724-4f09-42ea-aa89-832a14c93e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:27.035606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e5724-4f09-42ea-aa89-832a14c93e7d admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:27.035937 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e5724-4f09-42ea-aa89-832a14c93e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:27.036211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e5724-4f09-42ea-aa89-832a14c93e7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:27.036874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e5724-4f09-42ea-aa89-832a14c93e7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:27.038774 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2e5724-4f09-42ea-aa89-832a14c93e7d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:27.107168 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1278/2572] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:32:26 2022] POST /identity/v3/projects => generated 389 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:27.111223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f7ef6-fc58-456a-bbff-3713bbb1b0ba None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:27.122852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f7ef6-fc58-456a-bbff-3713bbb1b0ba None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:27.181364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d02a1f-0c8b-4642-978b-5f67c613d16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:27.181988 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d02a1f-0c8b-4642-978b-5f67c613d16b admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:27.182264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d02a1f-0c8b-4642-978b-5f67c613d16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:27.182535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d02a1f-0c8b-4642-978b-5f67c613d16b admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:27.183175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d02a1f-0c8b-4642-978b-5f67c613d16b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:27.185139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d02a1f-0c8b-4642-978b-5f67c613d16b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:27.273175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1295/2573] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:32:27 2022] POST /identity/v3/users => generated 301 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:27.278452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e5724-4f09-42ea-aa89-832a14c93e7d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:27.290403 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e5724-4f09-42ea-aa89-832a14c93e7d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:27.296281 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5f72c-20ef-42fe-8fb9-ce05f5e6c36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:27.297839 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5f72c-20ef-42fe-8fb9-ce05f5e6c36d admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:27.298002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5f72c-20ef-42fe-8fb9-ce05f5e6c36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:27.298828 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5f72c-20ef-42fe-8fb9-ce05f5e6c36d admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:27.300352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5f72c-20ef-42fe-8fb9-ce05f5e6c36d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:27.302162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5f72c-20ef-42fe-8fb9-ce05f5e6c36d admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:27.314742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1279/2574] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:27.319135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d02a1f-0c8b-4642-978b-5f67c613d16b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:27.343794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d02a1f-0c8b-4642-978b-5f67c613d16b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:27.349049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582a81d-dc6e-4d55-8067-da2bdc66404e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:27.349488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582a81d-dc6e-4d55-8067-da2bdc66404e admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:27.349860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582a81d-dc6e-4d55-8067-da2bdc66404e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:27.350226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582a81d-dc6e-4d55-8067-da2bdc66404e admin admin] PATH_INFO: `/v3/projects/3137cc12351c4a22aa940ce6ee71cbf8/users/674340058b744592ade05cbfd3dd4127/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:27.367659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582a81d-dc6e-4d55-8067-da2bdc66404e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3137cc12351c4a22aa940ce6ee71cbf8, user_id=674340058b744592ade05cbfd3dd4127, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:27.371182 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7582a81d-dc6e-4d55-8067-da2bdc66404e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:27.477494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1296/2575] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:27 2022] PUT /identity/v3/projects/3137cc12351c4a22aa940ce6ee71cbf8/users/674340058b744592ade05cbfd3dd4127/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:27.481501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5f72c-20ef-42fe-8fb9-ce05f5e6c36d admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:27.491516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5f72c-20ef-42fe-8fb9-ce05f5e6c36d admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:27.554194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463efb1-0f68-4c4c-afe1-8141c8d50724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:27.554812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463efb1-0f68-4c4c-afe1-8141c8d50724 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:27.555289 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463efb1-0f68-4c4c-afe1-8141c8d50724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:27.555580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463efb1-0f68-4c4c-afe1-8141c8d50724 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:27.556277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463efb1-0f68-4c4c-afe1-8141c8d50724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:27.557879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463efb1-0f68-4c4c-afe1-8141c8d50724 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:27.572947 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1280/2576] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:27.574379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582a81d-dc6e-4d55-8067-da2bdc66404e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:27.587268 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7582a81d-dc6e-4d55-8067-da2bdc66404e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:27.592301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b3227-2f9a-46bd-b5a5-dbbcbab6ef9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:27.592965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b3227-2f9a-46bd-b5a5-dbbcbab6ef9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:27.593248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b3227-2f9a-46bd-b5a5-dbbcbab6ef9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:27.593526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b3227-2f9a-46bd-b5a5-dbbcbab6ef9f admin admin] PATH_INFO: `/v3/projects/3137cc12351c4a22aa940ce6ee71cbf8/users/674340058b744592ade05cbfd3dd4127/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:27.595608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b3227-2f9a-46bd-b5a5-dbbcbab6ef9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3137cc12351c4a22aa940ce6ee71cbf8, user_id=674340058b744592ade05cbfd3dd4127, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:27.598368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b3227-2f9a-46bd-b5a5-dbbcbab6ef9f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:27.616491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1297/2577] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:27 2022] PUT /identity/v3/projects/3137cc12351c4a22aa940ce6ee71cbf8/users/674340058b744592ade05cbfd3dd4127/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:27.629653 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f42e07-e52a-4099-a850-80d6cfdfca82 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:27.630339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f42e07-e52a-4099-a850-80d6cfdfca82 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:27.630835 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f42e07-e52a-4099-a850-80d6cfdfca82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:27.662069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-43f42e07-e52a-4099-a850-80d6cfdfca82 None None] MFA Rules not processed for user `674340058b744592ade05cbfd3dd4127`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:27.663731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-43f42e07-e52a-4099-a850-80d6cfdfca82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:27.745908 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1281/2578] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:27 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:27.758239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b3227-2f9a-46bd-b5a5-dbbcbab6ef9f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:27.760288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-394b3227-2f9a-46bd-b5a5-dbbcbab6ef9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:27.846399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:27.850363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b3227-2f9a-46bd-b5a5-dbbcbab6ef9f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:27.854745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c04c56-1b88-4443-8b1a-797a3f79e8f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:27.860005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c04c56-1b88-4443-8b1a-797a3f79e8f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:27.860005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c04c56-1b88-4443-8b1a-797a3f79e8f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:27.860005 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c04c56-1b88-4443-8b1a-797a3f79e8f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:27.867013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c04c56-1b88-4443-8b1a-797a3f79e8f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:27.869812 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c04c56-1b88-4443-8b1a-797a3f79e8f0 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:27.882263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1298/2579] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:27 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:28.057262 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f42e07-e52a-4099-a850-80d6cfdfca82 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:28.071177 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:28.139514 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f42e07-e52a-4099-a850-80d6cfdfca82 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:28.144256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:28.144906 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:28.146885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:28.147017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] PATH_INFO: `/v3/users/b4af8bf91e454377a69ecbd26a2999aa` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:28.148534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4af8bf91e454377a69ecbd26a2999aa)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:28.150256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:28.358738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4af8bf91e454377a69ecbd26a2999aa'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:28.471961 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4af8bf91e454377a69ecbd26a2999aa'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:28.472654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4af8bf91e454377a69ecbd26a2999aa'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:28.473269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4af8bf91e454377a69ecbd26a2999aa'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:28.481283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4af8bf91e454377a69ecbd26a2999aa'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:28.500036 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1282/2580] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:28 2022] DELETE /identity/v3/users/b4af8bf91e454377a69ecbd26a2999aa => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:28.504343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1239c03-1034-4be0-b3d0-4f85bb620106 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:28.504829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1239c03-1034-4be0-b3d0-4f85bb620106 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:28.504929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1239c03-1034-4be0-b3d0-4f85bb620106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:28.524339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b1239c03-1034-4be0-b3d0-4f85bb620106 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:28.525567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b1239c03-1034-4be0-b3d0-4f85bb620106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:28.593754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1299/2581] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:28.604568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:28.606191 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:28.683731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63053c4-4a08-4ea8-965e-b1175f0efc9b admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:28.687519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23dae3d-4462-40a4-b2ce-0cc1305f25b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:28.688253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dae3d-4462-40a4-b2ce-0cc1305f25b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:28.688569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dae3d-4462-40a4-b2ce-0cc1305f25b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:28.688846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23dae3d-4462-40a4-b2ce-0cc1305f25b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:28.698645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23dae3d-4462-40a4-b2ce-0cc1305f25b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:28.700121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23dae3d-4462-40a4-b2ce-0cc1305f25b7 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:28.710189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1283/2582] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:29.332926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1239c03-1034-4be0-b3d0-4f85bb620106 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:29.334477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b1239c03-1034-4be0-b3d0-4f85bb620106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:29.353148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1239c03-1034-4be0-b3d0-4f85bb620106 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:29.386053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:29.386719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:29.387001 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:29.387277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] PATH_INFO: `/v3/projects/c8163ad6b5274157bcc26bdc5c18f8be` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:29.388450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8163ad6b5274157bcc26bdc5c18f8be)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:29.390402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:29.517148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8163ad6b5274157bcc26bdc5c18f8be', 'request_id': 'req-5f49ccee-6874-4f32-9def-e28074bb0b3e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:29.522782 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8163ad6b5274157bcc26bdc5c18f8be', 'request_id': 'req-5f49ccee-6874-4f32-9def-e28074bb0b3e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:29.637061 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8163ad6b5274157bcc26bdc5c18f8be', 'request_id': 'req-5f49ccee-6874-4f32-9def-e28074bb0b3e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:29.649212 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] The token cache is being invalidate because project c8163ad6b5274157bcc26bdc5c18f8be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:29.649504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:29.650154 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:29.651648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1300/2583] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:29 2022] DELETE /identity/v3/projects/c8163ad6b5274157bcc26bdc5c18f8be => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:29.655721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23dae3d-4462-40a4-b2ce-0cc1305f25b7 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:29.666576 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23dae3d-4462-40a4-b2ce-0cc1305f25b7 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:29.698684 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:29.736957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4fad6-041e-4dde-b6dc-47281f47cbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:29.737811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4fad6-041e-4dde-b6dc-47281f47cbeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:29.738170 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4fad6-041e-4dde-b6dc-47281f47cbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:29.738516 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4fad6-041e-4dde-b6dc-47281f47cbeb admin admin] PATH_INFO: `/v3/users/674340058b744592ade05cbfd3dd4127` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:29.740050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4fad6-041e-4dde-b6dc-47281f47cbeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=674340058b744592ade05cbfd3dd4127)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:29.742303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4fad6-041e-4dde-b6dc-47281f47cbeb admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:29.961788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-e4c4fad6-041e-4dde-b6dc-47281f47cbeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '674340058b744592ade05cbfd3dd4127'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:30.044421 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-e4c4fad6-041e-4dde-b6dc-47281f47cbeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '674340058b744592ade05cbfd3dd4127'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:30.045056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-e4c4fad6-041e-4dde-b6dc-47281f47cbeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '674340058b744592ade05cbfd3dd4127'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:30.046196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-e4c4fad6-041e-4dde-b6dc-47281f47cbeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '674340058b744592ade05cbfd3dd4127'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:30.053888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-e4c4fad6-041e-4dde-b6dc-47281f47cbeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '674340058b744592ade05cbfd3dd4127'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:30.072525 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1284/2584] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:29 2022] DELETE /identity/v3/users/674340058b744592ade05cbfd3dd4127 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:30.553892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:30.554987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:30.573203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49ccee-6874-4f32-9def-e28074bb0b3e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:30.658485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:30.659273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:30.659650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:30.660050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] PATH_INFO: `/v3/projects/3137cc12351c4a22aa940ce6ee71cbf8` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:30.661562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3137cc12351c4a22aa940ce6ee71cbf8)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:30.663584 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:30.686249 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:30.846421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3137cc12351c4a22aa940ce6ee71cbf8', 'request_id': 'req-e38f3937-a8a9-42f0-b6da-b261a94d05f5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:30.853059 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3137cc12351c4a22aa940ce6ee71cbf8', 'request_id': 'req-e38f3937-a8a9-42f0-b6da-b261a94d05f5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:30.957564 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3137cc12351c4a22aa940ce6ee71cbf8', 'request_id': 'req-e38f3937-a8a9-42f0-b6da-b261a94d05f5'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:30.973369 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] The token cache is being invalidate because project 3137cc12351c4a22aa940ce6ee71cbf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:30.973661 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:30.974313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:30.976956 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1301/2585] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:30 2022] DELETE /identity/v3/projects/3137cc12351c4a22aa940ce6ee71cbf8 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:30.984023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9ca43-e24c-469c-b354-269987482a76 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:30.991691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9ca43-e24c-469c-b354-269987482a76 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:30.991691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9ca43-e24c-469c-b354-269987482a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:30.991946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:30.991946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:31.010106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-daa9ca43-e24c-469c-b354-269987482a76 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:31.011936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-daa9ca43-e24c-469c-b354-269987482a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:31.062670 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f3937-a8a9-42f0-b6da-b261a94d05f5 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:31.067725 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc7afbe-65b9-4251-9544-236adfb8f048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:31.068508 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7afbe-65b9-4251-9544-236adfb8f048 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:31.068932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7afbe-65b9-4251-9544-236adfb8f048 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:31.069243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc7afbe-65b9-4251-9544-236adfb8f048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:31.071478 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-1fc7afbe-65b9-4251-9544-236adfb8f048 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:31.083026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1285/2586] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:31.090646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ff631-4692-4ff2-a0f0-2603a2977d57 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:31.090948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ff631-4692-4ff2-a0f0-2603a2977d57 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:31.091220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ff631-4692-4ff2-a0f0-2603a2977d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:31.110719 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-0f8ff631-4692-4ff2-a0f0-2603a2977d57 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:31.111974 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0f8ff631-4692-4ff2-a0f0-2603a2977d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:31.120121 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1286/2587] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:31.124511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ff631-4692-4ff2-a0f0-2603a2977d57 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:31.133831 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ff631-4692-4ff2-a0f0-2603a2977d57 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:31.138531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bba57a-dd77-43d5-b5cd-8a5183e7412f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:31.139148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bba57a-dd77-43d5-b5cd-8a5183e7412f admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:31.139429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bba57a-dd77-43d5-b5cd-8a5183e7412f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:31.139704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bba57a-dd77-43d5-b5cd-8a5183e7412f admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:31.140377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bba57a-dd77-43d5-b5cd-8a5183e7412f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:32:31.140701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bba57a-dd77-43d5-b5cd-8a5183e7412f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:31.142909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bba57a-dd77-43d5-b5cd-8a5183e7412f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:31.146232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc7afbe-65b9-4251-9544-236adfb8f048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:31.147723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc7afbe-65b9-4251-9544-236adfb8f048 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:31.154697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1287/2588] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:32:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:32:31.158643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1302/2589] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:31.158773 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bba57a-dd77-43d5-b5cd-8a5183e7412f admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:31.168343 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bba57a-dd77-43d5-b5cd-8a5183e7412f admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:31.172992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa7c09-d732-4f77-9b32-331165834e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:31.173580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa7c09-d732-4f77-9b32-331165834e56 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:31.173863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa7c09-d732-4f77-9b32-331165834e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:31.174139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa7c09-d732-4f77-9b32-331165834e56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:31.174764 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa7c09-d732-4f77-9b32-331165834e56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:31.176761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa7c09-d732-4f77-9b32-331165834e56 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:31.261507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1288/2590] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:32:31 2022] POST /identity/v3/projects => generated 359 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:32:31.265767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc7afbe-65b9-4251-9544-236adfb8f048 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:31.276065 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc7afbe-65b9-4251-9544-236adfb8f048 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:31.327665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:31.335062 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf313-b743-4f89-a142-04e1ebbf5131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:31.335809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf313-b743-4f89-a142-04e1ebbf5131 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:31.336095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf313-b743-4f89-a142-04e1ebbf5131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:31.336363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf313-b743-4f89-a142-04e1ebbf5131 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:31.337044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdf313-b743-4f89-a142-04e1ebbf5131 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:31.338979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdf313-b743-4f89-a142-04e1ebbf5131 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:31.497686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1303/2591] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:32:31 2022] POST /identity/v3/users => generated 280 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:31.498901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa7c09-d732-4f77-9b32-331165834e56 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:31.509808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa7c09-d732-4f77-9b32-331165834e56 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:31.514513 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a28e4-36c3-4e16-90c0-f4069e73ac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:31.515108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a28e4-36c3-4e16-90c0-f4069e73ac78 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:31.515385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a28e4-36c3-4e16-90c0-f4069e73ac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:31.515791 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a28e4-36c3-4e16-90c0-f4069e73ac78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:31.516472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a28e4-36c3-4e16-90c0-f4069e73ac78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:31.518309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a28e4-36c3-4e16-90c0-f4069e73ac78 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:31.521541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:31.528798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1289/2592] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:31.532855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf313-b743-4f89-a142-04e1ebbf5131 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:31.543484 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf313-b743-4f89-a142-04e1ebbf5131 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:31.549066 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dfc37-77a0-4f98-add3-0d23d4724119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:31.551622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dfc37-77a0-4f98-add3-0d23d4724119 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:31.551622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dfc37-77a0-4f98-add3-0d23d4724119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:31.551622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dfc37-77a0-4f98-add3-0d23d4724119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:31.551622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dfc37-77a0-4f98-add3-0d23d4724119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:31.552550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dfc37-77a0-4f98-add3-0d23d4724119 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:31.562878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1304/2593] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:31.567963 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a28e4-36c3-4e16-90c0-f4069e73ac78 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:31.580073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a28e4-36c3-4e16-90c0-f4069e73ac78 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:31.585382 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28859015-d5fc-413d-8865-83aa22b09839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:31.586051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28859015-d5fc-413d-8865-83aa22b09839 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:31.586336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28859015-d5fc-413d-8865-83aa22b09839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:31.586890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28859015-d5fc-413d-8865-83aa22b09839 admin admin] PATH_INFO: `/v3/projects/f87b39727ef543c88e21911a285b31e7/users/395436ef40ee48d9b088b7ba5bb75688/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:31.604385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28859015-d5fc-413d-8865-83aa22b09839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f87b39727ef543c88e21911a285b31e7, user_id=395436ef40ee48d9b088b7ba5bb75688, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:31.607454 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28859015-d5fc-413d-8865-83aa22b09839 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:31.718484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1290/2594] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:31 2022] PUT /identity/v3/projects/f87b39727ef543c88e21911a285b31e7/users/395436ef40ee48d9b088b7ba5bb75688/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:31.738537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533f8b9-9ace-4a77-8e2d-e1725c5d9beb None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:31.738537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533f8b9-9ace-4a77-8e2d-e1725c5d9beb None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:31.738537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533f8b9-9ace-4a77-8e2d-e1725c5d9beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:31.758667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-a533f8b9-9ace-4a77-8e2d-e1725c5d9beb None None] MFA Rules not processed for user `395436ef40ee48d9b088b7ba5bb75688`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:31.759916 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a533f8b9-9ace-4a77-8e2d-e1725c5d9beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:31.830644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1305/2595] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:31 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:31.868334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28859015-d5fc-413d-8865-83aa22b09839 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:31.871248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-28859015-d5fc-413d-8865-83aa22b09839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:31.930373 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28859015-d5fc-413d-8865-83aa22b09839 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:31.934215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9194c2c-52f6-4881-863c-9f5548bca5d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:31.934821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194c2c-52f6-4881-863c-9f5548bca5d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:31.935100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194c2c-52f6-4881-863c-9f5548bca5d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:31.935375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194c2c-52f6-4881-863c-9f5548bca5d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:31.944231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9194c2c-52f6-4881-863c-9f5548bca5d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:31.945705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9194c2c-52f6-4881-863c-9f5548bca5d6 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:31.955338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1291/2596] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:31.971620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533f8b9-9ace-4a77-8e2d-e1725c5d9beb None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:32.034681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533f8b9-9ace-4a77-8e2d-e1725c5d9beb None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:32.039272 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:32.039946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:32.040227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:32.040504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] PATH_INFO: `/v3/users/395436ef40ee48d9b088b7ba5bb75688` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:32.041642 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] RBAC: Authorizing `identity:delete_user(user_id=395436ef40ee48d9b088b7ba5bb75688)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:32.043573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:32.253660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '395436ef40ee48d9b088b7ba5bb75688'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:32.351678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '395436ef40ee48d9b088b7ba5bb75688'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:32.352329 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '395436ef40ee48d9b088b7ba5bb75688'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:32.352967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '395436ef40ee48d9b088b7ba5bb75688'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:32.360979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '395436ef40ee48d9b088b7ba5bb75688'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:32.362889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:32.378490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1306/2597] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:31 2022] DELETE /identity/v3/users/395436ef40ee48d9b088b7ba5bb75688 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:32.383173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4a5e0-36ca-4c7b-80e0-4e663a0106aa None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:32.383620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4a5e0-36ca-4c7b-80e0-4e663a0106aa None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:32.383737 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b4a5e0-36ca-4c7b-80e0-4e663a0106aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:32.403622 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-66b4a5e0-36ca-4c7b-80e0-4e663a0106aa None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:32.407630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-66b4a5e0-36ca-4c7b-80e0-4e663a0106aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:32.471415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1292/2598] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:32.481452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:32.482813 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:32.542707 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ac170-42b9-4bc9-8f5e-64142933535f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:32.547157 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663a66d-9997-4adf-a026-9d7f5e619bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:32.547862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663a66d-9997-4adf-a026-9d7f5e619bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:32.548145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663a66d-9997-4adf-a026-9d7f5e619bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:32.548424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663a66d-9997-4adf-a026-9d7f5e619bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:32.557486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663a66d-9997-4adf-a026-9d7f5e619bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:32.558917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663a66d-9997-4adf-a026-9d7f5e619bc5 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:32.569261 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1307/2599] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:33.033469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4a5e0-36ca-4c7b-80e0-4e663a0106aa None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:33.037132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-66b4a5e0-36ca-4c7b-80e0-4e663a0106aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:33.047031 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b4a5e0-36ca-4c7b-80e0-4e663a0106aa None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:33.056915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:33.057447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:33.057725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:33.058288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] PATH_INFO: `/v3/projects/f87b39727ef543c88e21911a285b31e7` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:33.059750 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f87b39727ef543c88e21911a285b31e7)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:33.065600 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:33.202171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f87b39727ef543c88e21911a285b31e7', 'request_id': 'req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:33.317684 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f87b39727ef543c88e21911a285b31e7', 'request_id': 'req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:33.327235 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f87b39727ef543c88e21911a285b31e7', 'request_id': 'req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:33.350365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:33.361114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] The token cache is being invalidate because project f87b39727ef543c88e21911a285b31e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:33.361728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:33.362077 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-221dc85d-3d4e-4dd5-a37d-da43c70d7b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:33.364017 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1293/2600] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:33 2022] DELETE /identity/v3/projects/f87b39727ef543c88e21911a285b31e7 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:33.369931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bd77a-f843-465c-98ad-5998d831ee5d None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:33.370299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bd77a-f843-465c-98ad-5998d831ee5d None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:33.370656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44bd77a-f843-465c-98ad-5998d831ee5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:33.401352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b44bd77a-f843-465c-98ad-5998d831ee5d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:33.402700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b44bd77a-f843-465c-98ad-5998d831ee5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:33.487754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1308/2601] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:33.489492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971d37c-5002-4768-aa58-896d45d1f042 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:33.489933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971d37c-5002-4768-aa58-896d45d1f042 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:33.490043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971d37c-5002-4768-aa58-896d45d1f042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:33.518001 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-f971d37c-5002-4768-aa58-896d45d1f042 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:33.519152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f971d37c-5002-4768-aa58-896d45d1f042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:33.527654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1294/2602] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:33.532580 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bd77a-f843-465c-98ad-5998d831ee5d None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:33.543457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44bd77a-f843-465c-98ad-5998d831ee5d None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:33.549801 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1980ff-5e28-4652-b142-f19a2c60d9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:33.550427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1980ff-5e28-4652-b142-f19a2c60d9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:33.550709 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1980ff-5e28-4652-b142-f19a2c60d9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:33.550982 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1980ff-5e28-4652-b142-f19a2c60d9e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:33.551635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1980ff-5e28-4652-b142-f19a2c60d9e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:32:33.551974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1980ff-5e28-4652-b142-f19a2c60d9e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:33.553586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1980ff-5e28-4652-b142-f19a2c60d9e5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:33.570419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1309/2603] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:32:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:32:33.575302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971d37c-5002-4768-aa58-896d45d1f042 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:33.586326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971d37c-5002-4768-aa58-896d45d1f042 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:33.593029 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395e716-3e78-49f6-afce-87c059b81292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:33.593709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395e716-3e78-49f6-afce-87c059b81292 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:33.594176 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395e716-3e78-49f6-afce-87c059b81292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:33.594970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395e716-3e78-49f6-afce-87c059b81292 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:33.595630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395e716-3e78-49f6-afce-87c059b81292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:33.597581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8395e716-3e78-49f6-afce-87c059b81292 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:33.713629 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1295/2604] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:32:33 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:33.717978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1980ff-5e28-4652-b142-f19a2c60d9e5 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:33.729170 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1980ff-5e28-4652-b142-f19a2c60d9e5 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:33.795379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3615e-aef1-47f6-a344-ab7d5b1e142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:33.796043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3615e-aef1-47f6-a344-ab7d5b1e142a admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:33.796326 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3615e-aef1-47f6-a344-ab7d5b1e142a admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:33.796605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a3615e-aef1-47f6-a344-ab7d5b1e142a admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:33.797254 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3615e-aef1-47f6-a344-ab7d5b1e142a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:33.799518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a3615e-aef1-47f6-a344-ab7d5b1e142a admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:33.921409 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1310/2605] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:32:33 2022] POST /identity/v3/users => generated 283 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:32:33.925178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395e716-3e78-49f6-afce-87c059b81292 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:33.935683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8395e716-3e78-49f6-afce-87c059b81292 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:33.940621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c782ee-770d-435c-a832-372fa00bc54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:33.941226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c782ee-770d-435c-a832-372fa00bc54c admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:33.941505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c782ee-770d-435c-a832-372fa00bc54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:33.941778 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c782ee-770d-435c-a832-372fa00bc54c admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:33.942419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c782ee-770d-435c-a832-372fa00bc54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:33.944081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c782ee-770d-435c-a832-372fa00bc54c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:33.955546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1296/2606] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:32:33.958760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3615e-aef1-47f6-a344-ab7d5b1e142a admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:33.964082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:33.969573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a3615e-aef1-47f6-a344-ab7d5b1e142a admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:33.974983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc96225-c9d4-493b-9712-184deaf93f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:33.975492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc96225-c9d4-493b-9712-184deaf93f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:33.975796 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc96225-c9d4-493b-9712-184deaf93f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:33.976055 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc96225-c9d4-493b-9712-184deaf93f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:33.976696 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc96225-c9d4-493b-9712-184deaf93f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:33.978458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc96225-c9d4-493b-9712-184deaf93f83 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:33.989694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1311/2607] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:32:33.996346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c782ee-770d-435c-a832-372fa00bc54c admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:34.008928 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c782ee-770d-435c-a832-372fa00bc54c admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:34.039958 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7383be-7546-48b8-912e-cb38b8be8261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:34.040655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7383be-7546-48b8-912e-cb38b8be8261 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:34.040938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7383be-7546-48b8-912e-cb38b8be8261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:34.041219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7383be-7546-48b8-912e-cb38b8be8261 admin admin] PATH_INFO: `/v3/projects/74b26eb5a72d4f06b5628b915b2f67ea/users/df5d0dadd3cd44678f632550e50ba77d/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:34.089537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7383be-7546-48b8-912e-cb38b8be8261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74b26eb5a72d4f06b5628b915b2f67ea, user_id=df5d0dadd3cd44678f632550e50ba77d, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:34.093103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7383be-7546-48b8-912e-cb38b8be8261 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:34.209043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1297/2608] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:32:33 2022] PUT /identity/v3/projects/74b26eb5a72d4f06b5628b915b2f67ea/users/df5d0dadd3cd44678f632550e50ba77d/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:34.210643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ba82c-5618-40e1-8b76-ae493f075da7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:34.210938 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ba82c-5618-40e1-8b76-ae493f075da7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:34.211553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ba82c-5618-40e1-8b76-ae493f075da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:34.243890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-c24ba82c-5618-40e1-8b76-ae493f075da7 None None] MFA Rules not processed for user `df5d0dadd3cd44678f632550e50ba77d`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:34.245219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c24ba82c-5618-40e1-8b76-ae493f075da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:34.317129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1312/2609] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:32:34.328427 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7383be-7546-48b8-912e-cb38b8be8261 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:34.335582 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-dd7383be-7546-48b8-912e-cb38b8be8261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:34.401675 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7383be-7546-48b8-912e-cb38b8be8261 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:34.415968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636016f8-7300-4532-b147-3806bb9b0a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:34.416602 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636016f8-7300-4532-b147-3806bb9b0a99 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:34.416883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636016f8-7300-4532-b147-3806bb9b0a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:34.417184 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636016f8-7300-4532-b147-3806bb9b0a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:34.429404 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636016f8-7300-4532-b147-3806bb9b0a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:34.430867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636016f8-7300-4532-b147-3806bb9b0a99 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:34.442083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1298/2610] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:32:37.404450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640a35-adf3-4c91-9c4b-43387e5ec87b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:37.409315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640a35-adf3-4c91-9c4b-43387e5ec87b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:37.409315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640a35-adf3-4c91-9c4b-43387e5ec87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:37.424889 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-9c640a35-adf3-4c91-9c4b-43387e5ec87b None None] MFA Rules not processed for user `537431860e3d4c4fa629254f8c4c0cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:32:37.426523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9c640a35-adf3-4c91-9c4b-43387e5ec87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:37.487797 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1313/2611] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:32:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:32:39.913405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636016f8-7300-4532-b147-3806bb9b0a99 service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:39.914780 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-636016f8-7300-4532-b147-3806bb9b0a99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:39.999483 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636016f8-7300-4532-b147-3806bb9b0a99 service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:40.005866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:40.006541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:40.006827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:40.007100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] PATH_INFO: `/v3/users/22402b1d1d9d489888883e1d2c7c60b5` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:40.008332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22402b1d1d9d489888883e1d2c7c60b5)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:40.010316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:40.177332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:40.182910 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22402b1d1d9d489888883e1d2c7c60b5'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:40.231400 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22402b1d1d9d489888883e1d2c7c60b5'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:40.232056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22402b1d1d9d489888883e1d2c7c60b5'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:40.232700 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22402b1d1d9d489888883e1d2c7c60b5'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:40.240125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22402b1d1d9d489888883e1d2c7c60b5'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:40.259100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1299/2612] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:39 2022] DELETE /identity/v3/users/22402b1d1d9d489888883e1d2c7c60b5 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:40.478862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c640a35-adf3-4c91-9c4b-43387e5ec87b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:40.492433 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c640a35-adf3-4c91-9c4b-43387e5ec87b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:40.563871 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:40.564761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:40.565041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:40.565317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] PATH_INFO: `/v3/projects/3b54810a2eb44e48a9ae72d8b5e1b121` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:40.566444 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b54810a2eb44e48a9ae72d8b5e1b121)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:40.568421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:40.708720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:40.723570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b54810a2eb44e48a9ae72d8b5e1b121', 'request_id': 'req-4d384718-b192-493f-bd5b-c97fdbaea781'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:40.728829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b54810a2eb44e48a9ae72d8b5e1b121', 'request_id': 'req-4d384718-b192-493f-bd5b-c97fdbaea781'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:40.819501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b54810a2eb44e48a9ae72d8b5e1b121', 'request_id': 'req-4d384718-b192-493f-bd5b-c97fdbaea781'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:40.832456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] The token cache is being invalidate because project 3b54810a2eb44e48a9ae72d8b5e1b121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:40.832751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:40.833494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:40.835028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1314/2613] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:40 2022] DELETE /identity/v3/projects/3b54810a2eb44e48a9ae72d8b5e1b121 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:40.839191 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:40.840618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:40.854457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31921e12-666c-4d5b-aabb-8e1abab8e8d3 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:40.911249 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:40.911910 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:40.912188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:40.912485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] PATH_INFO: `/v3/users/93f32e35eb0149a1a7aa3ab630022376` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:40.913615 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f32e35eb0149a1a7aa3ab630022376)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:40.915592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:41.092012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f32e35eb0149a1a7aa3ab630022376'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:41.145911 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f32e35eb0149a1a7aa3ab630022376'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:41.146527 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f32e35eb0149a1a7aa3ab630022376'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:41.147163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f32e35eb0149a1a7aa3ab630022376'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:41.154697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f32e35eb0149a1a7aa3ab630022376'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:41.173431 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1300/2614] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:32:40 2022] DELETE /identity/v3/users/93f32e35eb0149a1a7aa3ab630022376 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:41.410511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:41.411939 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:41.424902 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d384718-b192-493f-bd5b-c97fdbaea781 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:41.491305 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:41.492090 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:41.492463 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:41.492852 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] PATH_INFO: `/v3/projects/9b236904f41e453583d1c07ac3391af1` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:41.494064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b236904f41e453583d1c07ac3391af1)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:41.496192 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:41.625210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b236904f41e453583d1c07ac3391af1', 'request_id': 'req-eda7e5f5-8097-41da-beb6-186b5023b3f4'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:41.631499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b236904f41e453583d1c07ac3391af1', 'request_id': 'req-eda7e5f5-8097-41da-beb6-186b5023b3f4'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:41.797651 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b236904f41e453583d1c07ac3391af1', 'request_id': 'req-eda7e5f5-8097-41da-beb6-186b5023b3f4'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:41.809996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] The token cache is being invalidate because project 9b236904f41e453583d1c07ac3391af1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:32:41.810756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:41.811111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:32:41.812614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1315/2615] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:32:41 2022] DELETE /identity/v3/projects/9b236904f41e453583d1c07ac3391af1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:32:51.169621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=67,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:51.806930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=65,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:32:55.862236 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:32:55.864953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:55.949679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e203b-7222-41d5-b20d-4825b3566b79 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:32:56.027163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075413e0-b4b6-45fe-b5e5-88bf64a02416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:32:56.028132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075413e0-b4b6-45fe-b5e5-88bf64a02416 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:32:56.028596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075413e0-b4b6-45fe-b5e5-88bf64a02416 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:32:56.029498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075413e0-b4b6-45fe-b5e5-88bf64a02416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:32:56.032276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-075413e0-b4b6-45fe-b5e5-88bf64a02416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:32:56.120315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075413e0-b4b6-45fe-b5e5-88bf64a02416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:32:56.125742 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075413e0-b4b6-45fe-b5e5-88bf64a02416 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:32:56.132738 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1301/2616] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:32:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:01.916160 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:01.917655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:01.929752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7e5f5-8097-41da-beb6-186b5023b3f4 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:01.933332 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddca79d-8389-45f8-9979-9026ed082e2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:01.933921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddca79d-8389-45f8-9979-9026ed082e2c service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:01.934225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddca79d-8389-45f8-9979-9026ed082e2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:01.934494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddca79d-8389-45f8-9979-9026ed082e2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:01.936145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-4ddca79d-8389-45f8-9979-9026ed082e2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:02.003122 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddca79d-8389-45f8-9979-9026ed082e2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:02.004604 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddca79d-8389-45f8-9979-9026ed082e2c service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:02.015182 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1316/2617] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:05.702336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075413e0-b4b6-45fe-b5e5-88bf64a02416 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:05.704531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-075413e0-b4b6-45fe-b5e5-88bf64a02416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:05.719821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075413e0-b4b6-45fe-b5e5-88bf64a02416 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:05.725092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38501b70-7f9c-45ba-b5ee-b74d01b749de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:05.725790 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38501b70-7f9c-45ba-b5ee-b74d01b749de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:05.726070 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38501b70-7f9c-45ba-b5ee-b74d01b749de admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:05.726340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38501b70-7f9c-45ba-b5ee-b74d01b749de admin admin] PATH_INFO: `/v3/users/8430cbddaa1448f6ade09a082fa29b13` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:05.740842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38501b70-7f9c-45ba-b5ee-b74d01b749de admin admin] RBAC: Authorizing `identity:delete_user(user_id=8430cbddaa1448f6ade09a082fa29b13)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:05.742762 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38501b70-7f9c-45ba-b5ee-b74d01b749de admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:05.954565 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-38501b70-7f9c-45ba-b5ee-b74d01b749de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8430cbddaa1448f6ade09a082fa29b13'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:05.957188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:06.031495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-38501b70-7f9c-45ba-b5ee-b74d01b749de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8430cbddaa1448f6ade09a082fa29b13'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:06.032123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-38501b70-7f9c-45ba-b5ee-b74d01b749de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8430cbddaa1448f6ade09a082fa29b13'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:06.032724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-38501b70-7f9c-45ba-b5ee-b74d01b749de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8430cbddaa1448f6ade09a082fa29b13'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:06.040783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-38501b70-7f9c-45ba-b5ee-b74d01b749de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8430cbddaa1448f6ade09a082fa29b13'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:06.059830 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1302/2618] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:33:05 2022] DELETE /identity/v3/users/8430cbddaa1448f6ade09a082fa29b13 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:06.302566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddca79d-8389-45f8-9979-9026ed082e2c service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:06.316498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddca79d-8389-45f8-9979-9026ed082e2c service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:06.381685 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:06.382399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:06.382679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:06.382954 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] PATH_INFO: `/v3/projects/3d598cb3f17e407ab65dcdb8dbe3af8d` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:06.395506 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d598cb3f17e407ab65dcdb8dbe3af8d)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:06.397457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:06.506331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d598cb3f17e407ab65dcdb8dbe3af8d', 'request_id': 'req-65c5d347-20e4-4c48-a565-4b4d0c6c6967'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:06.512474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d598cb3f17e407ab65dcdb8dbe3af8d', 'request_id': 'req-65c5d347-20e4-4c48-a565-4b4d0c6c6967'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:06.574879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d598cb3f17e407ab65dcdb8dbe3af8d', 'request_id': 'req-65c5d347-20e4-4c48-a565-4b4d0c6c6967'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:06.587203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] The token cache is being invalidate because project 3d598cb3f17e407ab65dcdb8dbe3af8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:33:06.587486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:06.588148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-65c5d347-20e4-4c48-a565-4b4d0c6c6967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:06.589833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1317/2619] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:33:06 2022] DELETE /identity/v3/projects/3d598cb3f17e407ab65dcdb8dbe3af8d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:06.595390 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f00135-3e14-46a7-977b-5afa9af00cb5 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:06.595680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f00135-3e14-46a7-977b-5afa9af00cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:06.596041 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f00135-3e14-46a7-977b-5afa9af00cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:06.616306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-21f00135-3e14-46a7-977b-5afa9af00cb5 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:06.617505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-21f00135-3e14-46a7-977b-5afa9af00cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:06.678909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1303/2620] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:06.686897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d19a-aefb-47d6-9c7a-04d8bee1bd7b None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:06.687185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d19a-aefb-47d6-9c7a-04d8bee1bd7b None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:06.687485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4d19a-aefb-47d6-9c7a-04d8bee1bd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:06.707774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b8a4d19a-aefb-47d6-9c7a-04d8bee1bd7b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:06.708983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b8a4d19a-aefb-47d6-9c7a-04d8bee1bd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:06.716669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1318/2621] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:06.722644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f00135-3e14-46a7-977b-5afa9af00cb5 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:06.732391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f00135-3e14-46a7-977b-5afa9af00cb5 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:06.736893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39076028-3633-472e-a9e4-00800af85427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:06.737478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39076028-3633-472e-a9e4-00800af85427 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:06.737752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39076028-3633-472e-a9e4-00800af85427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:06.738020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39076028-3633-472e-a9e4-00800af85427 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:06.738652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39076028-3633-472e-a9e4-00800af85427 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:33:06.738951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39076028-3633-472e-a9e4-00800af85427 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:06.740565 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39076028-3633-472e-a9e4-00800af85427 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:06.751870 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1304/2622] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:33:06.756094 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4d19a-aefb-47d6-9c7a-04d8bee1bd7b None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:06.765223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4d19a-aefb-47d6-9c7a-04d8bee1bd7b None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:06.770315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6577dab-29aa-45ab-be92-bea622664217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:06.770905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6577dab-29aa-45ab-be92-bea622664217 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:06.771182 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6577dab-29aa-45ab-be92-bea622664217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:06.771453 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6577dab-29aa-45ab-be92-bea622664217 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:06.772118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6577dab-29aa-45ab-be92-bea622664217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:06.773987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6577dab-29aa-45ab-be92-bea622664217 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:06.779196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:06.878438 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1319/2623] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:06 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:06.882608 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39076028-3633-472e-a9e4-00800af85427 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:06.893178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39076028-3633-472e-a9e4-00800af85427 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:06.950953 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bfc29-7ee8-444c-a7db-567857cc033a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:06.951575 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bfc29-7ee8-444c-a7db-567857cc033a admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:06.952305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bfc29-7ee8-444c-a7db-567857cc033a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:06.952441 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bfc29-7ee8-444c-a7db-567857cc033a admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:06.953084 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4bfc29-7ee8-444c-a7db-567857cc033a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:06.955862 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4bfc29-7ee8-444c-a7db-567857cc033a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:07.066888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1305/2624] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:06 2022] POST /identity/v3/users => generated 283 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:07.071057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6577dab-29aa-45ab-be92-bea622664217 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:07.082179 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6577dab-29aa-45ab-be92-bea622664217 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:07.086821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3db1ee-f82d-4c76-985c-abbf0d7d3461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:07.087540 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3db1ee-f82d-4c76-985c-abbf0d7d3461 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:07.087857 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3db1ee-f82d-4c76-985c-abbf0d7d3461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:07.088133 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3db1ee-f82d-4c76-985c-abbf0d7d3461 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:07.088766 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3db1ee-f82d-4c76-985c-abbf0d7d3461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:07.090502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3db1ee-f82d-4c76-985c-abbf0d7d3461 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:07.100496 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1320/2625] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:07.104315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bfc29-7ee8-444c-a7db-567857cc033a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:07.114252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bfc29-7ee8-444c-a7db-567857cc033a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:07.118930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c6f33-0dc5-43c7-a9b4-7311a1f44317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:07.119691 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c6f33-0dc5-43c7-a9b4-7311a1f44317 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:07.119816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c6f33-0dc5-43c7-a9b4-7311a1f44317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:07.120082 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c6f33-0dc5-43c7-a9b4-7311a1f44317 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:07.120714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c6f33-0dc5-43c7-a9b4-7311a1f44317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:07.122300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c6f33-0dc5-43c7-a9b4-7311a1f44317 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:07.133720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1306/2626] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:07.137736 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3db1ee-f82d-4c76-985c-abbf0d7d3461 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:07.148342 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3db1ee-f82d-4c76-985c-abbf0d7d3461 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:07.152893 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ed98f-1aca-4a2b-bbdd-89a97fae86aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:07.153526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ed98f-1aca-4a2b-bbdd-89a97fae86aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:07.153799 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ed98f-1aca-4a2b-bbdd-89a97fae86aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:07.154069 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980ed98f-1aca-4a2b-bbdd-89a97fae86aa admin admin] PATH_INFO: `/v3/projects/9c02dc681d5a4e8fa95ef562cdcf646c/users/9ae8aba4aeb1489fb2d245d65f94dc42/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:07.168474 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ed98f-1aca-4a2b-bbdd-89a97fae86aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c02dc681d5a4e8fa95ef562cdcf646c, user_id=9ae8aba4aeb1489fb2d245d65f94dc42, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:07.172863 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980ed98f-1aca-4a2b-bbdd-89a97fae86aa admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:07.248358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1321/2627] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:07 2022] PUT /identity/v3/projects/9c02dc681d5a4e8fa95ef562cdcf646c/users/9ae8aba4aeb1489fb2d245d65f94dc42/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:07.260541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afb7a0-8111-4881-836b-925e17c06ba4 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:07.260827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afb7a0-8111-4881-836b-925e17c06ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:07.261103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afb7a0-8111-4881-836b-925e17c06ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:07.289083 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-25afb7a0-8111-4881-836b-925e17c06ba4 None None] MFA Rules not processed for user `9ae8aba4aeb1489fb2d245d65f94dc42`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:07.290365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-25afb7a0-8111-4881-836b-925e17c06ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:07.310116 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:07.358383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1307/2628] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:07.362959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ed98f-1aca-4a2b-bbdd-89a97fae86aa admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:07.423631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980ed98f-1aca-4a2b-bbdd-89a97fae86aa admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:07.428165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb79748-f81d-46a5-a6fc-232eaf8d6723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:07.428759 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb79748-f81d-46a5-a6fc-232eaf8d6723 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:07.429034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb79748-f81d-46a5-a6fc-232eaf8d6723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:07.429306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb79748-f81d-46a5-a6fc-232eaf8d6723 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:07.429933 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb79748-f81d-46a5-a6fc-232eaf8d6723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:07.431832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb79748-f81d-46a5-a6fc-232eaf8d6723 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:07.529674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1322/2629] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:07 2022] POST /identity/v3/projects => generated 365 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:07.534309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afb7a0-8111-4881-836b-925e17c06ba4 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:07.545061 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afb7a0-8111-4881-836b-925e17c06ba4 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:07.600549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1126d-344a-4bee-9e9e-abc5b3636c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:07.601143 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1126d-344a-4bee-9e9e-abc5b3636c6e admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:07.601485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1126d-344a-4bee-9e9e-abc5b3636c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:07.601696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1126d-344a-4bee-9e9e-abc5b3636c6e admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:07.602320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1126d-344a-4bee-9e9e-abc5b3636c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:07.604256 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1126d-344a-4bee-9e9e-abc5b3636c6e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:07.689172 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1308/2630] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:07 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:33:07.693238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb79748-f81d-46a5-a6fc-232eaf8d6723 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:07.703822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb79748-f81d-46a5-a6fc-232eaf8d6723 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:07.708363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee7cea-7f98-4c36-9a9a-f4f6ddc18e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:07.708949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee7cea-7f98-4c36-9a9a-f4f6ddc18e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:07.709224 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee7cea-7f98-4c36-9a9a-f4f6ddc18e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:07.709491 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee7cea-7f98-4c36-9a9a-f4f6ddc18e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:07.710117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee7cea-7f98-4c36-9a9a-f4f6ddc18e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:07.711695 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee7cea-7f98-4c36-9a9a-f4f6ddc18e67 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:07.722934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1323/2631] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:07.726806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1126d-344a-4bee-9e9e-abc5b3636c6e admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:07.736599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1126d-344a-4bee-9e9e-abc5b3636c6e admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:07.741152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a095174-2eae-47e9-9131-ddd08ca89ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:07.741792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a095174-2eae-47e9-9131-ddd08ca89ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:07.742192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a095174-2eae-47e9-9131-ddd08ca89ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:07.742469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a095174-2eae-47e9-9131-ddd08ca89ff9 admin admin] PATH_INFO: `/v3/projects/dda05a30358443649049f07ec44feaa0/users/23dd654dac41459bb6ec84fa3ab1b0fa/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:07.757796 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a095174-2eae-47e9-9131-ddd08ca89ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda05a30358443649049f07ec44feaa0, user_id=23dd654dac41459bb6ec84fa3ab1b0fa, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:07.760480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a095174-2eae-47e9-9131-ddd08ca89ff9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:07.802650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1309/2632] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:07 2022] PUT /identity/v3/projects/dda05a30358443649049f07ec44feaa0/users/23dd654dac41459bb6ec84fa3ab1b0fa/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:07.806620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee7cea-7f98-4c36-9a9a-f4f6ddc18e67 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:07.817397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee7cea-7f98-4c36-9a9a-f4f6ddc18e67 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:07.871743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63397595-e924-402a-96da-acdd20230318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:07.872414 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63397595-e924-402a-96da-acdd20230318 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:07.872694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63397595-e924-402a-96da-acdd20230318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:07.872970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63397595-e924-402a-96da-acdd20230318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:07.873598 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63397595-e924-402a-96da-acdd20230318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:07.875690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63397595-e924-402a-96da-acdd20230318 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:07.885211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1324/2633] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:07.889319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a095174-2eae-47e9-9131-ddd08ca89ff9 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:07.900108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a095174-2eae-47e9-9131-ddd08ca89ff9 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:07.904663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c54aa-8f16-46e6-b7ea-46d640b9a938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:07.905299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c54aa-8f16-46e6-b7ea-46d640b9a938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:07.905573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c54aa-8f16-46e6-b7ea-46d640b9a938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:07.905844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c54aa-8f16-46e6-b7ea-46d640b9a938 admin admin] PATH_INFO: `/v3/projects/dda05a30358443649049f07ec44feaa0/users/23dd654dac41459bb6ec84fa3ab1b0fa/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:07.907755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c54aa-8f16-46e6-b7ea-46d640b9a938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda05a30358443649049f07ec44feaa0, user_id=23dd654dac41459bb6ec84fa3ab1b0fa, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:07.910374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c54aa-8f16-46e6-b7ea-46d640b9a938 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:07.920245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1310/2634] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:07 2022] PUT /identity/v3/projects/dda05a30358443649049f07ec44feaa0/users/23dd654dac41459bb6ec84fa3ab1b0fa/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:07.930064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae3ede-a6b0-44d0-86a3-d36c38854864 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:07.930380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae3ede-a6b0-44d0-86a3-d36c38854864 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:07.930689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae3ede-a6b0-44d0-86a3-d36c38854864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:07.960998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-84ae3ede-a6b0-44d0-86a3-d36c38854864 None None] MFA Rules not processed for user `23dd654dac41459bb6ec84fa3ab1b0fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:07.962147 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-84ae3ede-a6b0-44d0-86a3-d36c38854864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:07.971241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:08.031050 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1325/2635] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:08.042257 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c54aa-8f16-46e6-b7ea-46d640b9a938 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:08.043989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7e5c54aa-8f16-46e6-b7ea-46d640b9a938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:08.101696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c54aa-8f16-46e6-b7ea-46d640b9a938 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:08.105408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664db1a-7482-4d4f-966f-89fe27cd04f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:08.106009 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664db1a-7482-4d4f-966f-89fe27cd04f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:08.106287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664db1a-7482-4d4f-966f-89fe27cd04f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:08.106561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664db1a-7482-4d4f-966f-89fe27cd04f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:08.115314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664db1a-7482-4d4f-966f-89fe27cd04f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:08.116761 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9664db1a-7482-4d4f-966f-89fe27cd04f3 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:08.126520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1311/2636] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:09.020307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae3ede-a6b0-44d0-86a3-d36c38854864 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:09.082964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae3ede-a6b0-44d0-86a3-d36c38854864 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:09.087541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:09.088210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:09.088488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:09.088762 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] PATH_INFO: `/v3/users/9ae8aba4aeb1489fb2d245d65f94dc42` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:09.089872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae8aba4aeb1489fb2d245d65f94dc42)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:09.091872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:09.291343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae8aba4aeb1489fb2d245d65f94dc42'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:09.368700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae8aba4aeb1489fb2d245d65f94dc42'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:09.369349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae8aba4aeb1489fb2d245d65f94dc42'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:09.369961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae8aba4aeb1489fb2d245d65f94dc42'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:09.378331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae8aba4aeb1489fb2d245d65f94dc42'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:09.396216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1326/2637] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:33:09 2022] DELETE /identity/v3/users/9ae8aba4aeb1489fb2d245d65f94dc42 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:09.400655 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6140-5a92-40c9-8c35-d940fa5f43a7 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:09.401123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6140-5a92-40c9-8c35-d940fa5f43a7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:09.401223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb6140-5a92-40c9-8c35-d940fa5f43a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:09.424975 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-82bb6140-5a92-40c9-8c35-d940fa5f43a7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:09.426306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-82bb6140-5a92-40c9-8c35-d940fa5f43a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:09.439178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:09.485867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1312/2638] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:09.495426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:09.496865 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:09.528079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:09.553904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563f9f4-12c0-4fa8-8e73-18b0f1ea8798 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:09.557949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dde74-1aa5-4591-b3d7-20bc3fd305c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:09.558589 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dde74-1aa5-4591-b3d7-20bc3fd305c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:09.558867 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dde74-1aa5-4591-b3d7-20bc3fd305c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:09.559142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0dde74-1aa5-4591-b3d7-20bc3fd305c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:09.568608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0dde74-1aa5-4591-b3d7-20bc3fd305c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:09.569997 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0dde74-1aa5-4591-b3d7-20bc3fd305c1 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:09.579747 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1327/2639] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:09.993338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb6140-5a92-40c9-8c35-d940fa5f43a7 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:09.994885 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-82bb6140-5a92-40c9-8c35-d940fa5f43a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:10.008259 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb6140-5a92-40c9-8c35-d940fa5f43a7 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:10.013613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:10.014420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:10.014711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:10.014991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] PATH_INFO: `/v3/projects/9c02dc681d5a4e8fa95ef562cdcf646c` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:10.016153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c02dc681d5a4e8fa95ef562cdcf646c)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:10.018220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:10.140309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c02dc681d5a4e8fa95ef562cdcf646c', 'request_id': 'req-34991ad3-f986-4c6d-b81a-80b07a3eaef8'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:10.202470 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c02dc681d5a4e8fa95ef562cdcf646c', 'request_id': 'req-34991ad3-f986-4c6d-b81a-80b07a3eaef8'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:10.208919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c02dc681d5a4e8fa95ef562cdcf646c', 'request_id': 'req-34991ad3-f986-4c6d-b81a-80b07a3eaef8'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:10.219914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] The token cache is being invalidate because project 9c02dc681d5a4e8fa95ef562cdcf646c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:33:10.220226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:10.220850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:10.222322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1313/2640] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:33:09 2022] DELETE /identity/v3/projects/9c02dc681d5a4e8fa95ef562cdcf646c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:10.226477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dde74-1aa5-4591-b3d7-20bc3fd305c1 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:10.237063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0dde74-1aa5-4591-b3d7-20bc3fd305c1 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:10.292114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21936e4-a1e6-42f4-af7e-4c69a902c8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:10.292734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21936e4-a1e6-42f4-af7e-4c69a902c8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:10.293014 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21936e4-a1e6-42f4-af7e-4c69a902c8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:10.293288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21936e4-a1e6-42f4-af7e-4c69a902c8b9 admin admin] PATH_INFO: `/v3/users/23dd654dac41459bb6ec84fa3ab1b0fa` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:10.294396 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21936e4-a1e6-42f4-af7e-4c69a902c8b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23dd654dac41459bb6ec84fa3ab1b0fa)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:10.296333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21936e4-a1e6-42f4-af7e-4c69a902c8b9 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:10.513673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e21936e4-a1e6-42f4-af7e-4c69a902c8b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23dd654dac41459bb6ec84fa3ab1b0fa'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:10.602923 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e21936e4-a1e6-42f4-af7e-4c69a902c8b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23dd654dac41459bb6ec84fa3ab1b0fa'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:10.603543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e21936e4-a1e6-42f4-af7e-4c69a902c8b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23dd654dac41459bb6ec84fa3ab1b0fa'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:10.604185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e21936e4-a1e6-42f4-af7e-4c69a902c8b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23dd654dac41459bb6ec84fa3ab1b0fa'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:10.611436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e21936e4-a1e6-42f4-af7e-4c69a902c8b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23dd654dac41459bb6ec84fa3ab1b0fa'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:10.628550 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1328/2641] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:33:10 2022] DELETE /identity/v3/users/23dd654dac41459bb6ec84fa3ab1b0fa => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:11.107448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:11.108990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:11.122583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34991ad3-f986-4c6d-b81a-80b07a3eaef8 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:11.180580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:11.181213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:11.181500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:11.181777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] PATH_INFO: `/v3/projects/dda05a30358443649049f07ec44feaa0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:11.182894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda05a30358443649049f07ec44feaa0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:11.184880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:11.268489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:11.271989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda05a30358443649049f07ec44feaa0', 'request_id': 'req-150b6640-6d36-4496-a37c-67888482cbc1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:11.340355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda05a30358443649049f07ec44feaa0', 'request_id': 'req-150b6640-6d36-4496-a37c-67888482cbc1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:11.346890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda05a30358443649049f07ec44feaa0', 'request_id': 'req-150b6640-6d36-4496-a37c-67888482cbc1'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:11.357399 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] The token cache is being invalidate because project dda05a30358443649049f07ec44feaa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:33:11.357619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:11.358283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-150b6640-6d36-4496-a37c-67888482cbc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:11.360519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1314/2642] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:33:11 2022] DELETE /identity/v3/projects/dda05a30358443649049f07ec44feaa0 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:11.369514 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea737fce-8cd3-4de1-b76a-45ecb15a4da5 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:11.369991 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea737fce-8cd3-4de1-b76a-45ecb15a4da5 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:11.370110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea737fce-8cd3-4de1-b76a-45ecb15a4da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:11.390263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ea737fce-8cd3-4de1-b76a-45ecb15a4da5 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:11.394450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ea737fce-8cd3-4de1-b76a-45ecb15a4da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:11.434716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:11.450821 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1329/2643] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:11.458681 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b714d1d-1ff4-4d72-b0c5-dde95a3e8a84 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:11.458962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b714d1d-1ff4-4d72-b0c5-dde95a3e8a84 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:11.459233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b714d1d-1ff4-4d72-b0c5-dde95a3e8a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:11.479167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-4b714d1d-1ff4-4d72-b0c5-dde95a3e8a84 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:11.480440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-4b714d1d-1ff4-4d72-b0c5-dde95a3e8a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:11.489182 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1315/2644] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:11.493340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea737fce-8cd3-4de1-b76a-45ecb15a4da5 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:11.503609 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea737fce-8cd3-4de1-b76a-45ecb15a4da5 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:11.508239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2228e8-5b0c-44eb-ba24-a308c82a53a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:11.508834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2228e8-5b0c-44eb-ba24-a308c82a53a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:11.509114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2228e8-5b0c-44eb-ba24-a308c82a53a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:11.509389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2228e8-5b0c-44eb-ba24-a308c82a53a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:11.510035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2228e8-5b0c-44eb-ba24-a308c82a53a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:33:11.510560 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2228e8-5b0c-44eb-ba24-a308c82a53a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:11.512824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2228e8-5b0c-44eb-ba24-a308c82a53a7 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:11.524386 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1330/2645] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:33:11.528567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b714d1d-1ff4-4d72-b0c5-dde95a3e8a84 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:11.538990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b714d1d-1ff4-4d72-b0c5-dde95a3e8a84 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:11.543606 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d997ce-412b-415c-aba3-e4a9331ec367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:11.544240 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d997ce-412b-415c-aba3-e4a9331ec367 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:11.544535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d997ce-412b-415c-aba3-e4a9331ec367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:11.549379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d997ce-412b-415c-aba3-e4a9331ec367 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:11.549379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d997ce-412b-415c-aba3-e4a9331ec367 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:11.549379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d997ce-412b-415c-aba3-e4a9331ec367 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:11.626712 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1316/2646] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:11 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:11.631244 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2228e8-5b0c-44eb-ba24-a308c82a53a7 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:11.641538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2228e8-5b0c-44eb-ba24-a308c82a53a7 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:11.703576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee67014-8b10-4e85-a20b-1db6e4541949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:11.704380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee67014-8b10-4e85-a20b-1db6e4541949 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:11.704656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee67014-8b10-4e85-a20b-1db6e4541949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:11.704929 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee67014-8b10-4e85-a20b-1db6e4541949 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:11.705600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee67014-8b10-4e85-a20b-1db6e4541949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:11.707576 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee67014-8b10-4e85-a20b-1db6e4541949 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:11.809432 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1331/2647] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:11 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:11.813697 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d997ce-412b-415c-aba3-e4a9331ec367 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:11.824267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d997ce-412b-415c-aba3-e4a9331ec367 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:11.829124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872355aa-fe30-476f-a7cd-f9210763a9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:11.830021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872355aa-fe30-476f-a7cd-f9210763a9bb admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:11.830021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872355aa-fe30-476f-a7cd-f9210763a9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:11.830264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872355aa-fe30-476f-a7cd-f9210763a9bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:11.830895 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872355aa-fe30-476f-a7cd-f9210763a9bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:11.832553 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872355aa-fe30-476f-a7cd-f9210763a9bb admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:11.848278 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1317/2648] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:11.849504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee67014-8b10-4e85-a20b-1db6e4541949 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:11.863985 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee67014-8b10-4e85-a20b-1db6e4541949 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:11.873397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefb33d-a9b6-46ed-a18f-49d30b40b03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:11.873397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefb33d-a9b6-46ed-a18f-49d30b40b03b admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:11.873397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefb33d-a9b6-46ed-a18f-49d30b40b03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:11.873397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefb33d-a9b6-46ed-a18f-49d30b40b03b admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:11.873872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefb33d-a9b6-46ed-a18f-49d30b40b03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:11.873872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefb33d-a9b6-46ed-a18f-49d30b40b03b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:11.885516 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1332/2649] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:11.888592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872355aa-fe30-476f-a7cd-f9210763a9bb admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:11.899038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872355aa-fe30-476f-a7cd-f9210763a9bb admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:11.903663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa6341-9ba5-4037-a000-6897325f0578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:11.904335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa6341-9ba5-4037-a000-6897325f0578 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:11.904614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa6341-9ba5-4037-a000-6897325f0578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:11.904891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa6341-9ba5-4037-a000-6897325f0578 admin admin] PATH_INFO: `/v3/projects/60f152dd1a99419b9d68f8ca8919993d/users/053165bca13a42f48b08a19bf1b2fdc4/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:11.920055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa6341-9ba5-4037-a000-6897325f0578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f152dd1a99419b9d68f8ca8919993d, user_id=053165bca13a42f48b08a19bf1b2fdc4, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:11.922685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa6341-9ba5-4037-a000-6897325f0578 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:12.002305 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1318/2650] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:11 2022] PUT /identity/v3/projects/60f152dd1a99419b9d68f8ca8919993d/users/053165bca13a42f48b08a19bf1b2fdc4/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:12.013623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61d55e-3679-4282-a691-d2ba78a5bee5 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:12.013912 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61d55e-3679-4282-a691-d2ba78a5bee5 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:12.014190 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61d55e-3679-4282-a691-d2ba78a5bee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:12.042221 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-5b61d55e-3679-4282-a691-d2ba78a5bee5 None None] MFA Rules not processed for user `053165bca13a42f48b08a19bf1b2fdc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:12.043446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5b61d55e-3679-4282-a691-d2ba78a5bee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:12.106690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1333/2651] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:12.111134 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa6341-9ba5-4037-a000-6897325f0578 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:12.171863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa6341-9ba5-4037-a000-6897325f0578 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:12.176358 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24056b56-e14d-4f2a-ac3f-e37c374721a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:12.176952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24056b56-e14d-4f2a-ac3f-e37c374721a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:12.177231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24056b56-e14d-4f2a-ac3f-e37c374721a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:12.177502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24056b56-e14d-4f2a-ac3f-e37c374721a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:12.178126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24056b56-e14d-4f2a-ac3f-e37c374721a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:12.180043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24056b56-e14d-4f2a-ac3f-e37c374721a1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:12.260090 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1319/2652] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:12 2022] POST /identity/v3/projects => generated 367 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:12.264126 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61d55e-3679-4282-a691-d2ba78a5bee5 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:12.275349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61d55e-3679-4282-a691-d2ba78a5bee5 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:12.316088 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:12.331012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fb78a-ff1b-4b77-b88b-9a248271275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:12.331621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fb78a-ff1b-4b77-b88b-9a248271275f admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:12.331940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fb78a-ff1b-4b77-b88b-9a248271275f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:12.332218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17fb78a-ff1b-4b77-b88b-9a248271275f admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:12.332840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fb78a-ff1b-4b77-b88b-9a248271275f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:12.334730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17fb78a-ff1b-4b77-b88b-9a248271275f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:12.425694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1334/2653] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:12 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:12.429662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24056b56-e14d-4f2a-ac3f-e37c374721a1 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:12.441047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24056b56-e14d-4f2a-ac3f-e37c374721a1 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:12.445880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ced2cb-7f32-4304-b22d-78c22ac9976f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:12.446476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ced2cb-7f32-4304-b22d-78c22ac9976f admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:12.446760 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ced2cb-7f32-4304-b22d-78c22ac9976f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:12.447039 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ced2cb-7f32-4304-b22d-78c22ac9976f admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:12.447726 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ced2cb-7f32-4304-b22d-78c22ac9976f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:12.449343 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ced2cb-7f32-4304-b22d-78c22ac9976f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:12.461060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1320/2654] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:12.465012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fb78a-ff1b-4b77-b88b-9a248271275f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:12.474948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17fb78a-ff1b-4b77-b88b-9a248271275f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:12.479582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891f998-3cc9-4a2c-94d2-0cbd1eeec5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:12.480263 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891f998-3cc9-4a2c-94d2-0cbd1eeec5c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:12.480543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891f998-3cc9-4a2c-94d2-0cbd1eeec5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:12.480816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891f998-3cc9-4a2c-94d2-0cbd1eeec5c1 admin admin] PATH_INFO: `/v3/projects/a6668e7bcea546b3a89e9b2f58e6e293/users/f1e8d80e725742d59aabb02dcb2f5cb7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:12.495052 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891f998-3cc9-4a2c-94d2-0cbd1eeec5c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6668e7bcea546b3a89e9b2f58e6e293, user_id=f1e8d80e725742d59aabb02dcb2f5cb7, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:12.497732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7891f998-3cc9-4a2c-94d2-0cbd1eeec5c1 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:12.557194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1335/2655] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:12 2022] PUT /identity/v3/projects/a6668e7bcea546b3a89e9b2f58e6e293/users/f1e8d80e725742d59aabb02dcb2f5cb7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:12.561049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ced2cb-7f32-4304-b22d-78c22ac9976f admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:12.571956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ced2cb-7f32-4304-b22d-78c22ac9976f admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:12.610722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:12.626854 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ed766-0132-4523-bab3-c07c5b0d8f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:12.627444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ed766-0132-4523-bab3-c07c5b0d8f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:12.627719 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ed766-0132-4523-bab3-c07c5b0d8f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:12.628027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ed766-0132-4523-bab3-c07c5b0d8f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:12.628652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ed766-0132-4523-bab3-c07c5b0d8f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:12.630262 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ed766-0132-4523-bab3-c07c5b0d8f2e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:12.639741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1321/2656] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:12.644616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891f998-3cc9-4a2c-94d2-0cbd1eeec5c1 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:12.654729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891f998-3cc9-4a2c-94d2-0cbd1eeec5c1 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:12.659352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc3639-e79f-4183-a4b7-e1cf8ec1ecc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:12.660101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc3639-e79f-4183-a4b7-e1cf8ec1ecc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:12.664299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc3639-e79f-4183-a4b7-e1cf8ec1ecc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:12.664299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc3639-e79f-4183-a4b7-e1cf8ec1ecc0 admin admin] PATH_INFO: `/v3/projects/a6668e7bcea546b3a89e9b2f58e6e293/users/f1e8d80e725742d59aabb02dcb2f5cb7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:12.664299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc3639-e79f-4183-a4b7-e1cf8ec1ecc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6668e7bcea546b3a89e9b2f58e6e293, user_id=f1e8d80e725742d59aabb02dcb2f5cb7, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:12.665266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc3639-e79f-4183-a4b7-e1cf8ec1ecc0 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:12.676024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1336/2657] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:12 2022] PUT /identity/v3/projects/a6668e7bcea546b3a89e9b2f58e6e293/users/f1e8d80e725742d59aabb02dcb2f5cb7/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:12.685861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209719c-26f1-4bbc-a82f-b366f8c20a55 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:12.687378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209719c-26f1-4bbc-a82f-b366f8c20a55 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:12.687378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209719c-26f1-4bbc-a82f-b366f8c20a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:12.714205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-f209719c-26f1-4bbc-a82f-b366f8c20a55 None None] MFA Rules not processed for user `f1e8d80e725742d59aabb02dcb2f5cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:12.715402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f209719c-26f1-4bbc-a82f-b366f8c20a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:12.784350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1322/2658] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:12.795591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc3639-e79f-4183-a4b7-e1cf8ec1ecc0 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:12.797837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-3fbc3639-e79f-4183-a4b7-e1cf8ec1ecc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:12.854017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc3639-e79f-4183-a4b7-e1cf8ec1ecc0 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:12.857622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db3e90-502a-4344-ad4e-080c68cf688a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:12.858216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db3e90-502a-4344-ad4e-080c68cf688a service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:12.858488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db3e90-502a-4344-ad4e-080c68cf688a service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:12.858760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db3e90-502a-4344-ad4e-080c68cf688a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:12.867919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db3e90-502a-4344-ad4e-080c68cf688a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:12.869323 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db3e90-502a-4344-ad4e-080c68cf688a service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:12.878926 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1337/2659] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:12.989488 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209719c-26f1-4bbc-a82f-b366f8c20a55 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:12.990893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f209719c-26f1-4bbc-a82f-b366f8c20a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:13.003539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209719c-26f1-4bbc-a82f-b366f8c20a55 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:13.007261 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af9279-7ac5-43fa-b5a9-5f500b4b29df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:13.007884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af9279-7ac5-43fa-b5a9-5f500b4b29df service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:13.008163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af9279-7ac5-43fa-b5a9-5f500b4b29df service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:13.008432 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af9279-7ac5-43fa-b5a9-5f500b4b29df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:13.066948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af9279-7ac5-43fa-b5a9-5f500b4b29df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:13.068423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af9279-7ac5-43fa-b5a9-5f500b4b29df service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:13.077239 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1323/2660] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:17.824123 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db3e90-502a-4344-ad4e-080c68cf688a service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:17.884835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db3e90-502a-4344-ad4e-080c68cf688a service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:17.889356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:17.889976 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:17.890255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:17.891037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] PATH_INFO: `/v3/users/053165bca13a42f48b08a19bf1b2fdc4` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:17.892163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=053165bca13a42f48b08a19bf1b2fdc4)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:17.894068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:18.028672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:18.034404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '053165bca13a42f48b08a19bf1b2fdc4'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:18.077178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '053165bca13a42f48b08a19bf1b2fdc4'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:18.077761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '053165bca13a42f48b08a19bf1b2fdc4'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:18.078903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '053165bca13a42f48b08a19bf1b2fdc4'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:18.085868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '053165bca13a42f48b08a19bf1b2fdc4'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:18.104225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1338/2661] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:33:17 2022] DELETE /identity/v3/users/053165bca13a42f48b08a19bf1b2fdc4 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:18.108813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84d136-3db9-43e5-bc5d-27726fb3d7b7 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:18.114952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84d136-3db9-43e5-bc5d-27726fb3d7b7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:18.114952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84d136-3db9-43e5-bc5d-27726fb3d7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:18.128287 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-3b84d136-3db9-43e5-bc5d-27726fb3d7b7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:18.129479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3b84d136-3db9-43e5-bc5d-27726fb3d7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:18.189721 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1324/2662] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:18.200007 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:18.201513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:18.260011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076e7e23-df5c-4c25-8f26-9be9201d18a5 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:18.263608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4cb56-8f17-495b-bf09-a4b67343fec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:18.264240 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4cb56-8f17-495b-bf09-a4b67343fec0 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:18.264518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4cb56-8f17-495b-bf09-a4b67343fec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:18.264790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4cb56-8f17-495b-bf09-a4b67343fec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:18.274219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4cb56-8f17-495b-bf09-a4b67343fec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:18.275629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4cb56-8f17-495b-bf09-a4b67343fec0 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:18.285019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1339/2663] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:18.691014 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84d136-3db9-43e5-bc5d-27726fb3d7b7 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:18.692462 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3b84d136-3db9-43e5-bc5d-27726fb3d7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:18.699620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:18.706147 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84d136-3db9-43e5-bc5d-27726fb3d7b7 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:18.710928 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:18.711567 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:18.711956 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:18.712231 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] PATH_INFO: `/v3/projects/60f152dd1a99419b9d68f8ca8919993d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:18.713362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] RBAC: Authorizing `identity:delete_project(project_id=60f152dd1a99419b9d68f8ca8919993d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:18.715266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:18.814917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60f152dd1a99419b9d68f8ca8919993d', 'request_id': 'req-32a09433-58a8-4da6-8265-2e0f4b7ad23c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:18.963008 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60f152dd1a99419b9d68f8ca8919993d', 'request_id': 'req-32a09433-58a8-4da6-8265-2e0f4b7ad23c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:18.964012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60f152dd1a99419b9d68f8ca8919993d', 'request_id': 'req-32a09433-58a8-4da6-8265-2e0f4b7ad23c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:18.974668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] The token cache is being invalidate because project 60f152dd1a99419b9d68f8ca8919993d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:33:18.982050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:18.982050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:18.982050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1325/2664] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:33:18 2022] DELETE /identity/v3/projects/60f152dd1a99419b9d68f8ca8919993d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:18.985460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4cb56-8f17-495b-bf09-a4b67343fec0 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:18.995068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4cb56-8f17-495b-bf09-a4b67343fec0 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:19.050768 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc5fef0-b3e7-4b1c-b3f4-03d2cbbe0f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:19.053582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5fef0-b3e7-4b1c-b3f4-03d2cbbe0f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:19.053582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5fef0-b3e7-4b1c-b3f4-03d2cbbe0f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:19.053582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5fef0-b3e7-4b1c-b3f4-03d2cbbe0f7d admin admin] PATH_INFO: `/v3/users/f1e8d80e725742d59aabb02dcb2f5cb7` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:19.053582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5fef0-b3e7-4b1c-b3f4-03d2cbbe0f7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e8d80e725742d59aabb02dcb2f5cb7)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:19.054961 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc5fef0-b3e7-4b1c-b3f4-03d2cbbe0f7d admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:19.176523 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6dc5fef0-b3e7-4b1c-b3f4-03d2cbbe0f7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e8d80e725742d59aabb02dcb2f5cb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:19.236673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6dc5fef0-b3e7-4b1c-b3f4-03d2cbbe0f7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e8d80e725742d59aabb02dcb2f5cb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:19.237352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6dc5fef0-b3e7-4b1c-b3f4-03d2cbbe0f7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e8d80e725742d59aabb02dcb2f5cb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:19.238006 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6dc5fef0-b3e7-4b1c-b3f4-03d2cbbe0f7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e8d80e725742d59aabb02dcb2f5cb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:19.245694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6dc5fef0-b3e7-4b1c-b3f4-03d2cbbe0f7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e8d80e725742d59aabb02dcb2f5cb7'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:19.248280 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:19.263945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1340/2665] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:33:18 2022] DELETE /identity/v3/users/f1e8d80e725742d59aabb02dcb2f5cb7 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:19.701132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:19.702601 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:19.716850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a09433-58a8-4da6-8265-2e0f4b7ad23c admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:19.775248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:19.775921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:19.776204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:19.776481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] PATH_INFO: `/v3/projects/a6668e7bcea546b3a89e9b2f58e6e293` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:19.777592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6668e7bcea546b3a89e9b2f58e6e293)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:19.779573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:19.883026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6668e7bcea546b3a89e9b2f58e6e293', 'request_id': 'req-c6e56953-2316-41fd-bea9-e438b806a57f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:19.950999 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6668e7bcea546b3a89e9b2f58e6e293', 'request_id': 'req-c6e56953-2316-41fd-bea9-e438b806a57f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:19.957660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6668e7bcea546b3a89e9b2f58e6e293', 'request_id': 'req-c6e56953-2316-41fd-bea9-e438b806a57f'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:19.967979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] The token cache is being invalidate because project a6668e7bcea546b3a89e9b2f58e6e293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:33:19.968265 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:19.969430 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c6e56953-2316-41fd-bea9-e438b806a57f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:19.970896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1326/2666] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:33:19 2022] DELETE /identity/v3/projects/a6668e7bcea546b3a89e9b2f58e6e293 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:19.975901 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f58e8-ed72-4893-9be3-6d68f8d788e8 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:19.976136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f58e8-ed72-4893-9be3-6d68f8d788e8 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:19.976421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9f58e8-ed72-4893-9be3-6d68f8d788e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:19.996693 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6a9f58e8-ed72-4893-9be3-6d68f8d788e8 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:19.997942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6a9f58e8-ed72-4893-9be3-6d68f8d788e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:20.056958 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1341/2667] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:20.065141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ecd6c-74d0-495b-aea6-1c987d89cf89 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:20.065692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ecd6c-74d0-495b-aea6-1c987d89cf89 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:20.065967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ecd6c-74d0-495b-aea6-1c987d89cf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:20.085211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-3b1ecd6c-74d0-495b-aea6-1c987d89cf89 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:20.086402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3b1ecd6c-74d0-495b-aea6-1c987d89cf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:20.095020 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1327/2668] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:20.099215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f58e8-ed72-4893-9be3-6d68f8d788e8 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:20.108869 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9f58e8-ed72-4893-9be3-6d68f8d788e8 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:20.113953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d733f-3eeb-4cfb-9fba-765b2a6c8ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:20.114548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d733f-3eeb-4cfb-9fba-765b2a6c8ade admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:20.114823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d733f-3eeb-4cfb-9fba-765b2a6c8ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:20.115095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d733f-3eeb-4cfb-9fba-765b2a6c8ade admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:20.115727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d733f-3eeb-4cfb-9fba-765b2a6c8ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:33:20.116063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d733f-3eeb-4cfb-9fba-765b2a6c8ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:20.117660 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646d733f-3eeb-4cfb-9fba-765b2a6c8ade admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:20.128822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1342/2669] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:33:20.132838 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ecd6c-74d0-495b-aea6-1c987d89cf89 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:20.142412 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ecd6c-74d0-495b-aea6-1c987d89cf89 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:20.146950 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8265f3c-7eff-4356-960e-c1ca0c841317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:20.147530 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8265f3c-7eff-4356-960e-c1ca0c841317 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:20.147828 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8265f3c-7eff-4356-960e-c1ca0c841317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:20.148102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8265f3c-7eff-4356-960e-c1ca0c841317 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:20.148740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8265f3c-7eff-4356-960e-c1ca0c841317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:20.150611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8265f3c-7eff-4356-960e-c1ca0c841317 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:20.155807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:20.221132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1328/2670] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:20 2022] POST /identity/v3/projects => generated 367 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:33:20.225427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d733f-3eeb-4cfb-9fba-765b2a6c8ade admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:20.235553 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d733f-3eeb-4cfb-9fba-765b2a6c8ade admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:20.292888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b635ae0-7980-4df7-8f6a-3f75d2909848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:20.293494 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b635ae0-7980-4df7-8f6a-3f75d2909848 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:20.293771 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b635ae0-7980-4df7-8f6a-3f75d2909848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:20.294041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b635ae0-7980-4df7-8f6a-3f75d2909848 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:20.294664 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b635ae0-7980-4df7-8f6a-3f75d2909848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:20.296588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b635ae0-7980-4df7-8f6a-3f75d2909848 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:20.357403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1343/2671] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:20 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:20.361680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8265f3c-7eff-4356-960e-c1ca0c841317 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:20.372082 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8265f3c-7eff-4356-960e-c1ca0c841317 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:20.376631 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004cfe7-b326-4e98-a830-527a47559032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:20.377224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004cfe7-b326-4e98-a830-527a47559032 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:20.377501 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004cfe7-b326-4e98-a830-527a47559032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:20.377800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004cfe7-b326-4e98-a830-527a47559032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:20.378457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004cfe7-b326-4e98-a830-527a47559032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:20.380074 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004cfe7-b326-4e98-a830-527a47559032 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:20.390230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1329/2672] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:20.394890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b635ae0-7980-4df7-8f6a-3f75d2909848 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:20.404840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b635ae0-7980-4df7-8f6a-3f75d2909848 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:20.409997 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d6bd4-0041-4fb8-b57e-4e98abf1b564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:20.410582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6bd4-0041-4fb8-b57e-4e98abf1b564 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:20.410856 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6bd4-0041-4fb8-b57e-4e98abf1b564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:20.411125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d6bd4-0041-4fb8-b57e-4e98abf1b564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:20.411752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d6bd4-0041-4fb8-b57e-4e98abf1b564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:20.413399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d6bd4-0041-4fb8-b57e-4e98abf1b564 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:20.423698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1344/2673] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:20.427642 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004cfe7-b326-4e98-a830-527a47559032 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:20.437962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004cfe7-b326-4e98-a830-527a47559032 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:20.442853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05df22-ce10-427a-acb5-5b6bb0fc83d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:20.443497 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05df22-ce10-427a-acb5-5b6bb0fc83d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:20.443786 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05df22-ce10-427a-acb5-5b6bb0fc83d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:20.444080 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05df22-ce10-427a-acb5-5b6bb0fc83d1 admin admin] PATH_INFO: `/v3/projects/2649312e7d5d41f38be74410d821bc3b/users/7453d4a307b141b6bb970a6cdcf6d365/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:20.459148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05df22-ce10-427a-acb5-5b6bb0fc83d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2649312e7d5d41f38be74410d821bc3b, user_id=7453d4a307b141b6bb970a6cdcf6d365, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:20.461851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e05df22-ce10-427a-acb5-5b6bb0fc83d1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:20.573989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1330/2674] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:20 2022] PUT /identity/v3/projects/2649312e7d5d41f38be74410d821bc3b/users/7453d4a307b141b6bb970a6cdcf6d365/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:20.585767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff246-9540-4053-b016-d9461195c2ff None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:20.586037 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff246-9540-4053-b016-d9461195c2ff None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:20.586306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ff246-9540-4053-b016-d9461195c2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:20.613194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-2a1ff246-9540-4053-b016-d9461195c2ff None None] MFA Rules not processed for user `7453d4a307b141b6bb970a6cdcf6d365`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:20.614400 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2a1ff246-9540-4053-b016-d9461195c2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:20.647726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:20.677115 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1345/2675] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:20.681102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05df22-ce10-427a-acb5-5b6bb0fc83d1 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:20.742586 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e05df22-ce10-427a-acb5-5b6bb0fc83d1 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:20.747268 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fe1db-ab5d-496c-9bbd-88d4688e46a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:20.747896 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fe1db-ab5d-496c-9bbd-88d4688e46a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:20.748178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fe1db-ab5d-496c-9bbd-88d4688e46a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:20.748452 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5fe1db-ab5d-496c-9bbd-88d4688e46a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:20.749072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fe1db-ab5d-496c-9bbd-88d4688e46a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:20.751492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5fe1db-ab5d-496c-9bbd-88d4688e46a2 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:20.855857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1331/2676] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:20 2022] POST /identity/v3/projects => generated 367 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:20.860667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ff246-9540-4053-b016-d9461195c2ff None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:20.871485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ff246-9540-4053-b016-d9461195c2ff None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:20.928056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc63cf-1287-4e6f-9e01-24bd8b77bb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:20.928662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc63cf-1287-4e6f-9e01-24bd8b77bb21 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:20.928940 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc63cf-1287-4e6f-9e01-24bd8b77bb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:20.929210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc63cf-1287-4e6f-9e01-24bd8b77bb21 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:20.929827 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc63cf-1287-4e6f-9e01-24bd8b77bb21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:20.931730 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc63cf-1287-4e6f-9e01-24bd8b77bb21 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:20.996729 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1346/2677] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:20 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:21.000715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fe1db-ab5d-496c-9bbd-88d4688e46a2 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:21.011295 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5fe1db-ab5d-496c-9bbd-88d4688e46a2 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:21.016693 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845ec89-219c-4a35-bb6c-3b11833984d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:21.017296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845ec89-219c-4a35-bb6c-3b11833984d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:21.017579 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845ec89-219c-4a35-bb6c-3b11833984d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:21.017853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845ec89-219c-4a35-bb6c-3b11833984d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:21.018500 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845ec89-219c-4a35-bb6c-3b11833984d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:21.020143 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845ec89-219c-4a35-bb6c-3b11833984d2 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:21.030968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1332/2678] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:33:21.035582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc63cf-1287-4e6f-9e01-24bd8b77bb21 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:21.045483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc63cf-1287-4e6f-9e01-24bd8b77bb21 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:21.050183 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc4436-96bd-4ef5-b225-49722e2a7e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:21.050836 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc4436-96bd-4ef5-b225-49722e2a7e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:21.051116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc4436-96bd-4ef5-b225-49722e2a7e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:21.051396 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc4436-96bd-4ef5-b225-49722e2a7e2f admin admin] PATH_INFO: `/v3/projects/39eb377c8ae04d96abde878305330896/users/724e97e37fd94fb28bc2d88b05242ec3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:21.066336 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc4436-96bd-4ef5-b225-49722e2a7e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eb377c8ae04d96abde878305330896, user_id=724e97e37fd94fb28bc2d88b05242ec3, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:21.069035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc4436-96bd-4ef5-b225-49722e2a7e2f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:21.127626 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1347/2679] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:21 2022] PUT /identity/v3/projects/39eb377c8ae04d96abde878305330896/users/724e97e37fd94fb28bc2d88b05242ec3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:21.131473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845ec89-219c-4a35-bb6c-3b11833984d2 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:21.142092 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845ec89-219c-4a35-bb6c-3b11833984d2 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:21.198725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e464ce-f05b-4186-91fa-bcf45f261a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:21.199320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e464ce-f05b-4186-91fa-bcf45f261a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:21.199598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e464ce-f05b-4186-91fa-bcf45f261a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:21.199902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e464ce-f05b-4186-91fa-bcf45f261a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:21.200528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e464ce-f05b-4186-91fa-bcf45f261a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:21.202097 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e464ce-f05b-4186-91fa-bcf45f261a37 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:21.211980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1333/2680] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:21.216039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc4436-96bd-4ef5-b225-49722e2a7e2f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:21.226986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc4436-96bd-4ef5-b225-49722e2a7e2f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:21.231547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639410b-f854-4dea-97fc-3381f07dce9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:21.232226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639410b-f854-4dea-97fc-3381f07dce9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:21.232507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639410b-f854-4dea-97fc-3381f07dce9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:21.232783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d639410b-f854-4dea-97fc-3381f07dce9b admin admin] PATH_INFO: `/v3/projects/39eb377c8ae04d96abde878305330896/users/724e97e37fd94fb28bc2d88b05242ec3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:21.234678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639410b-f854-4dea-97fc-3381f07dce9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=39eb377c8ae04d96abde878305330896, user_id=724e97e37fd94fb28bc2d88b05242ec3, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:21.237339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d639410b-f854-4dea-97fc-3381f07dce9b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:21.247279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1348/2681] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:21 2022] PUT /identity/v3/projects/39eb377c8ae04d96abde878305330896/users/724e97e37fd94fb28bc2d88b05242ec3/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:21.257598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e62ca-4692-4929-ba9b-7162bba7c5e1 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:21.257882 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e62ca-4692-4929-ba9b-7162bba7c5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:21.258157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e62ca-4692-4929-ba9b-7162bba7c5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:21.285371 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-861e62ca-4692-4929-ba9b-7162bba7c5e1 None None] MFA Rules not processed for user `724e97e37fd94fb28bc2d88b05242ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:21.286565 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-861e62ca-4692-4929-ba9b-7162bba7c5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:21.296190 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:21.355471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1334/2682] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:21.371417 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639410b-f854-4dea-97fc-3381f07dce9b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:21.373266 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d639410b-f854-4dea-97fc-3381f07dce9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:21.431150 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d639410b-f854-4dea-97fc-3381f07dce9b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:21.434727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ca3f-eacd-48fd-aa24-dcf0132c7e0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:21.435324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ca3f-eacd-48fd-aa24-dcf0132c7e0a service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:21.435785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ca3f-eacd-48fd-aa24-dcf0132c7e0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:21.436032 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3ca3f-eacd-48fd-aa24-dcf0132c7e0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:21.446208 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3ca3f-eacd-48fd-aa24-dcf0132c7e0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:21.449851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3ca3f-eacd-48fd-aa24-dcf0132c7e0a service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:21.458017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1349/2683] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:21.697646 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e62ca-4692-4929-ba9b-7162bba7c5e1 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:21.710311 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e62ca-4692-4929-ba9b-7162bba7c5e1 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:21.714079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8afff10-2629-4955-9bef-c769966e7406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:21.714811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afff10-2629-4955-9bef-c769966e7406 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:21.715094 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afff10-2629-4955-9bef-c769966e7406 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:21.715366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8afff10-2629-4955-9bef-c769966e7406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:21.772658 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afff10-2629-4955-9bef-c769966e7406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:21.774095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8afff10-2629-4955-9bef-c769966e7406 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:21.783057 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1335/2684] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:27.971856 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ca3f-eacd-48fd-aa24-dcf0132c7e0a service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:28.005419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:28.034123 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3ca3f-eacd-48fd-aa24-dcf0132c7e0a service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:28.038597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:28.039213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:28.039489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:28.039756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] PATH_INFO: `/v3/users/7453d4a307b141b6bb970a6cdcf6d365` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:28.040871 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7453d4a307b141b6bb970a6cdcf6d365)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:28.042819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:28.219170 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7453d4a307b141b6bb970a6cdcf6d365'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:28.265313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7453d4a307b141b6bb970a6cdcf6d365'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:28.265931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7453d4a307b141b6bb970a6cdcf6d365'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:28.266534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7453d4a307b141b6bb970a6cdcf6d365'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:28.273874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7453d4a307b141b6bb970a6cdcf6d365'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:28.291538 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1350/2685] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:33:27 2022] DELETE /identity/v3/users/7453d4a307b141b6bb970a6cdcf6d365 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:28.296103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c44efd-a4c2-43eb-ad6d-0c11ada7f84a None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:28.296652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c44efd-a4c2-43eb-ad6d-0c11ada7f84a None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:28.296652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c44efd-a4c2-43eb-ad6d-0c11ada7f84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:28.316821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-46c44efd-a4c2-43eb-ad6d-0c11ada7f84a None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:28.318011 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-46c44efd-a4c2-43eb-ad6d-0c11ada7f84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:28.382194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1336/2686] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:28.395359 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:28.396932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:28.455567 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d7e007-4450-401c-907c-f209be743a68 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:28.463167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7131b8b-a175-4b59-939a-86af1043811f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:28.463167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7131b8b-a175-4b59-939a-86af1043811f service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:28.463167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7131b8b-a175-4b59-939a-86af1043811f service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:28.463167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7131b8b-a175-4b59-939a-86af1043811f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:28.471163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7131b8b-a175-4b59-939a-86af1043811f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:28.472744 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7131b8b-a175-4b59-939a-86af1043811f service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:28.482239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1351/2687] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:28.953507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c44efd-a4c2-43eb-ad6d-0c11ada7f84a None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:28.955958 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-46c44efd-a4c2-43eb-ad6d-0c11ada7f84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:28.978370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c44efd-a4c2-43eb-ad6d-0c11ada7f84a None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:28.985592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:28.985592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:28.985592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:28.985592 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] PATH_INFO: `/v3/projects/2649312e7d5d41f38be74410d821bc3b` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:28.987140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=2649312e7d5d41f38be74410d821bc3b)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:28.989160 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:29.135692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:29.141632 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2649312e7d5d41f38be74410d821bc3b', 'request_id': 'req-ac597d92-de65-4374-88e1-ef896c2976ab'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.231330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2649312e7d5d41f38be74410d821bc3b', 'request_id': 'req-ac597d92-de65-4374-88e1-ef896c2976ab'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.237949 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2649312e7d5d41f38be74410d821bc3b', 'request_id': 'req-ac597d92-de65-4374-88e1-ef896c2976ab'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.248636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] The token cache is being invalidate because project 2649312e7d5d41f38be74410d821bc3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:33:29.248919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.249534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.250983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1337/2688] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:33:28 2022] DELETE /identity/v3/projects/2649312e7d5d41f38be74410d821bc3b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:29.255431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7131b8b-a175-4b59-939a-86af1043811f service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:29.265253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7131b8b-a175-4b59-939a-86af1043811f service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:29.311810 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:29.321080 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb43734-441d-4ec9-a549-b790070bc3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:29.324743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb43734-441d-4ec9-a549-b790070bc3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:29.324743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb43734-441d-4ec9-a549-b790070bc3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:29.324743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb43734-441d-4ec9-a549-b790070bc3c5 admin admin] PATH_INFO: `/v3/users/724e97e37fd94fb28bc2d88b05242ec3` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:29.324743 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb43734-441d-4ec9-a549-b790070bc3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=724e97e37fd94fb28bc2d88b05242ec3)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:29.325260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb43734-441d-4ec9-a549-b790070bc3c5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:29.422350 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:29.423930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:29.436834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac597d92-de65-4374-88e1-ef896c2976ab admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:29.442581 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:29.443203 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:29.443478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:29.443747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] PATH_INFO: `/v3/users/df5d0dadd3cd44678f632550e50ba77d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:29.444922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df5d0dadd3cd44678f632550e50ba77d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:29.446841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:29.491161 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7eb43734-441d-4ec9-a549-b790070bc3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724e97e37fd94fb28bc2d88b05242ec3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.574501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7eb43734-441d-4ec9-a549-b790070bc3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724e97e37fd94fb28bc2d88b05242ec3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.575145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7eb43734-441d-4ec9-a549-b790070bc3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724e97e37fd94fb28bc2d88b05242ec3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.575757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7eb43734-441d-4ec9-a549-b790070bc3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724e97e37fd94fb28bc2d88b05242ec3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.582953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7eb43734-441d-4ec9-a549-b790070bc3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724e97e37fd94fb28bc2d88b05242ec3'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.600164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1352/2689] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:33:29 2022] DELETE /identity/v3/users/724e97e37fd94fb28bc2d88b05242ec3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:29.656677 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df5d0dadd3cd44678f632550e50ba77d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.780208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df5d0dadd3cd44678f632550e50ba77d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.780821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df5d0dadd3cd44678f632550e50ba77d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.781460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df5d0dadd3cd44678f632550e50ba77d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.788924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df5d0dadd3cd44678f632550e50ba77d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:29.806075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1338/2690] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:33:29 2022] DELETE /identity/v3/users/df5d0dadd3cd44678f632550e50ba77d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:33:29.810622 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5128c5-bd33-4d67-a99e-44ea75706f36 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:29.810904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5128c5-bd33-4d67-a99e-44ea75706f36 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:29.811175 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5128c5-bd33-4d67-a99e-44ea75706f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:29.957606 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ea5128c5-bd33-4d67-a99e-44ea75706f36 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:29.958830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ea5128c5-bd33-4d67-a99e-44ea75706f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:30.019680 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1353/2691] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:30.029414 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:30.032187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:30.077981 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5128c5-bd33-4d67-a99e-44ea75706f36 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:30.080288 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ea5128c5-bd33-4d67-a99e-44ea75706f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:30.093265 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5128c5-bd33-4d67-a99e-44ea75706f36 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:30.103552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:30.103552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.103552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.103552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] PATH_INFO: `/v3/projects/39eb377c8ae04d96abde878305330896` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.104308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39eb377c8ae04d96abde878305330896)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:30.104308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:30.106977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a5b95-984b-4d2c-9d67-b1e9928289d1 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:30.111138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6b2ab-7f29-4758-8342-8bad2ac81779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:30.111759 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6b2ab-7f29-4758-8342-8bad2ac81779 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.112060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6b2ab-7f29-4758-8342-8bad2ac81779 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.112332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6b2ab-7f29-4758-8342-8bad2ac81779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.121747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6b2ab-7f29-4758-8342-8bad2ac81779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:30.123200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6b2ab-7f29-4758-8342-8bad2ac81779 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:30.132866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1339/2692] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:30.198761 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39eb377c8ae04d96abde878305330896', 'request_id': 'req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:30.204283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39eb377c8ae04d96abde878305330896', 'request_id': 'req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:30.271042 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39eb377c8ae04d96abde878305330896', 'request_id': 'req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:30.273297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:30.283385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] The token cache is being invalidate because project 39eb377c8ae04d96abde878305330896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:33:30.283682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:30.284409 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7e7d06cc-f6fd-4f3f-8fee-b8df014c31e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:30.285866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1354/2693] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:33:30 2022] DELETE /identity/v3/projects/39eb377c8ae04d96abde878305330896 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:30.290884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b28d7-ee43-4703-9ff0-568979d5b44e None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.291171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b28d7-ee43-4703-9ff0-568979d5b44e None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.291447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b28d7-ee43-4703-9ff0-568979d5b44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.311447 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-6d7b28d7-ee43-4703-9ff0-568979d5b44e None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:30.312687 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6d7b28d7-ee43-4703-9ff0-568979d5b44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:30.357699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:30.379690 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1340/2694] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:30.397071 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb430025-80ec-4da7-b557-f43083123b34 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.397353 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb430025-80ec-4da7-b557-f43083123b34 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.397625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb430025-80ec-4da7-b557-f43083123b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.417896 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-cb430025-80ec-4da7-b557-f43083123b34 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:30.419196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-cb430025-80ec-4da7-b557-f43083123b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:30.427085 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1355/2695] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:30.431554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b28d7-ee43-4703-9ff0-568979d5b44e None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:30.442051 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b28d7-ee43-4703-9ff0-568979d5b44e None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:30.446652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc242768-2b0a-4858-932f-7573ffe12226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:30.447248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242768-2b0a-4858-932f-7573ffe12226 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.447526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242768-2b0a-4858-932f-7573ffe12226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.447823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242768-2b0a-4858-932f-7573ffe12226 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.448469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc242768-2b0a-4858-932f-7573ffe12226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:33:30.448769 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc242768-2b0a-4858-932f-7573ffe12226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:30.450354 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc242768-2b0a-4858-932f-7573ffe12226 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:30.466076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1341/2696] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:33:30.467539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb430025-80ec-4da7-b557-f43083123b34 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:30.478452 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb430025-80ec-4da7-b557-f43083123b34 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:30.483115 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f304b-20e4-40f0-a99c-a9df61e92b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:30.483712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f304b-20e4-40f0-a99c-a9df61e92b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.484017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f304b-20e4-40f0-a99c-a9df61e92b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.484286 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f304b-20e4-40f0-a99c-a9df61e92b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.484919 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f304b-20e4-40f0-a99c-a9df61e92b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:30.486824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f304b-20e4-40f0-a99c-a9df61e92b22 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:30.581997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc242768-2b0a-4858-932f-7573ffe12226 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:30.584298 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-dc242768-2b0a-4858-932f-7573ffe12226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:30.585160 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1356/2697] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:30 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:30.589558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f304b-20e4-40f0-a99c-a9df61e92b22 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:30.607724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc242768-2b0a-4858-932f-7573ffe12226 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:30.668330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f304b-20e4-40f0-a99c-a9df61e92b22 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:30.668989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:30.669645 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.669946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.670243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] PATH_INFO: `/v3/projects/74b26eb5a72d4f06b5628b915b2f67ea` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.671361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=74b26eb5a72d4f06b5628b915b2f67ea)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:30.673171 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2525dc-328b-4404-9106-bb38021af8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:30.673326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:30.673777 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2525dc-328b-4404-9106-bb38021af8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.674051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2525dc-328b-4404-9106-bb38021af8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.674325 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2525dc-328b-4404-9106-bb38021af8f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.674937 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2525dc-328b-4404-9106-bb38021af8f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:30.677604 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2525dc-328b-4404-9106-bb38021af8f6 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:30.775138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1357/2698] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:30 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:33:30.779029 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2525dc-328b-4404-9106-bb38021af8f6 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:30.788413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2525dc-328b-4404-9106-bb38021af8f6 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:30.793328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aab461-d6c0-4f68-a974-23555063da95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:30.797989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aab461-d6c0-4f68-a974-23555063da95 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.797989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aab461-d6c0-4f68-a974-23555063da95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.797989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aab461-d6c0-4f68-a974-23555063da95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.797989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aab461-d6c0-4f68-a974-23555063da95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:30.797989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aab461-d6c0-4f68-a974-23555063da95 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:30.806823 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1358/2699] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:30.810056 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74b26eb5a72d4f06b5628b915b2f67ea', 'request_id': 'req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:30.810614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aab461-d6c0-4f68-a974-23555063da95 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:30.821563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aab461-d6c0-4f68-a974-23555063da95 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:30.878206 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e36d5-52fd-424c-bea2-6dc14c5382ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:30.878815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e36d5-52fd-424c-bea2-6dc14c5382ea admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.879094 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e36d5-52fd-424c-bea2-6dc14c5382ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.879370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e36d5-52fd-424c-bea2-6dc14c5382ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.880074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e36d5-52fd-424c-bea2-6dc14c5382ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:30.881734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3e36d5-52fd-424c-bea2-6dc14c5382ea admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:30.891559 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1359/2700] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:33:30.896307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e36d5-52fd-424c-bea2-6dc14c5382ea admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:30.905498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74b26eb5a72d4f06b5628b915b2f67ea', 'request_id': 'req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:30.905784 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e36d5-52fd-424c-bea2-6dc14c5382ea admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:30.910541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb150bc2-9313-4934-99e2-7c8ca1cca978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:30.911179 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb150bc2-9313-4934-99e2-7c8ca1cca978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.911460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb150bc2-9313-4934-99e2-7c8ca1cca978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.911733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb150bc2-9313-4934-99e2-7c8ca1cca978 admin admin] PATH_INFO: `/v3/projects/04b40d23e6fe4be6970569b115034bc7/users/a269667f143243f5a60bf4cbb3096b09/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.913490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74b26eb5a72d4f06b5628b915b2f67ea', 'request_id': 'req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:30.924922 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] The token cache is being invalidate because project 74b26eb5a72d4f06b5628b915b2f67ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:33:30.925215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:30.925907 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-78f9edfc-0ed5-449a-b234-dcdabe59fcdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:30.927234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb150bc2-9313-4934-99e2-7c8ca1cca978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04b40d23e6fe4be6970569b115034bc7, user_id=a269667f143243f5a60bf4cbb3096b09, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:30.927482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1342/2701] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:33:30 2022] DELETE /identity/v3/projects/74b26eb5a72d4f06b5628b915b2f67ea => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:33:30.929932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb150bc2-9313-4934-99e2-7c8ca1cca978 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:30.932143 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ed765-1895-4eaf-b6a6-8b6f2985eaa9 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:30.932429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ed765-1895-4eaf-b6a6-8b6f2985eaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:30.932714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ed765-1895-4eaf-b6a6-8b6f2985eaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:30.954209 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-924ed765-1895-4eaf-b6a6-8b6f2985eaa9 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:30.956523 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-924ed765-1895-4eaf-b6a6-8b6f2985eaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:30.965715 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1343/2702] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:33:31.016163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1360/2703] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:30 2022] PUT /identity/v3/projects/04b40d23e6fe4be6970569b115034bc7/users/a269667f143243f5a60bf4cbb3096b09/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:33:31.037710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34410676-6dab-4ea5-ab89-4dbe0f82d4f0 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.037996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34410676-6dab-4ea5-ab89-4dbe0f82d4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.038107 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d0ff5-1dc8-4bfd-81c4-e22433aebcb1 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.038312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34410676-6dab-4ea5-ab89-4dbe0f82d4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.038419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d0ff5-1dc8-4bfd-81c4-e22433aebcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.038647 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d0ff5-1dc8-4bfd-81c4-e22433aebcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.060129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-5b0d0ff5-1dc8-4bfd-81c4-e22433aebcb1 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:31.061364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5b0d0ff5-1dc8-4bfd-81c4-e22433aebcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:31.068788 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-34410676-6dab-4ea5-ab89-4dbe0f82d4f0 None None] MFA Rules not processed for user `a269667f143243f5a60bf4cbb3096b09`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:31.069976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-34410676-6dab-4ea5-ab89-4dbe0f82d4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:31.074645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:31.114511 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:31.122875 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1361/2704] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:33:31.128019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d0ff5-1dc8-4bfd-81c4-e22433aebcb1 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.142301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1344/2705] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:31.145974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d0ff5-1dc8-4bfd-81c4-e22433aebcb1 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.146154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34410676-6dab-4ea5-ab89-4dbe0f82d4f0 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.147957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-34410676-6dab-4ea5-ab89-4dbe0f82d4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:31.157040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadad72c-2b9a-40bf-abe0-a3de8b1f06ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.157666 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadad72c-2b9a-40bf-abe0-a3de8b1f06ea admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.157963 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadad72c-2b9a-40bf-abe0-a3de8b1f06ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.158333 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadad72c-2b9a-40bf-abe0-a3de8b1f06ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.159360 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadad72c-2b9a-40bf-abe0-a3de8b1f06ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:33:31.159829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadad72c-2b9a-40bf-abe0-a3de8b1f06ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.162264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadad72c-2b9a-40bf-abe0-a3de8b1f06ea admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.163904 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34410676-6dab-4ea5-ab89-4dbe0f82d4f0 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.169132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad648cf-e92f-4b1a-acba-63bd4c49f3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.169725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad648cf-e92f-4b1a-acba-63bd4c49f3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.170002 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad648cf-e92f-4b1a-acba-63bd4c49f3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.170274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad648cf-e92f-4b1a-acba-63bd4c49f3c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.170928 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad648cf-e92f-4b1a-acba-63bd4c49f3c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.172851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad648cf-e92f-4b1a-acba-63bd4c49f3c1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.178357 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1362/2706] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:33:31.182544 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadad72c-2b9a-40bf-abe0-a3de8b1f06ea admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.193082 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadad72c-2b9a-40bf-abe0-a3de8b1f06ea admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.197600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ac497-63b7-4450-80e8-24089f1527a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.198229 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ac497-63b7-4450-80e8-24089f1527a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.198502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ac497-63b7-4450-80e8-24089f1527a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.198770 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ac497-63b7-4450-80e8-24089f1527a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.199415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ac497-63b7-4450-80e8-24089f1527a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.201368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ac497-63b7-4450-80e8-24089f1527a7 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.247087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1345/2707] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:31 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:33:31.251044 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad648cf-e92f-4b1a-acba-63bd4c49f3c1 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.316434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1363/2708] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:31 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:33:31.321162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ac497-63b7-4450-80e8-24089f1527a7 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.333640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad648cf-e92f-4b1a-acba-63bd4c49f3c1 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.392774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ac497-63b7-4450-80e8-24089f1527a7 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.393152 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ce1db-994d-41a6-9821-4bd734cb14ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.393758 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ce1db-994d-41a6-9821-4bd734cb14ad admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.394034 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ce1db-994d-41a6-9821-4bd734cb14ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.394303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ce1db-994d-41a6-9821-4bd734cb14ad admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.394929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ce1db-994d-41a6-9821-4bd734cb14ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.396943 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ce1db-994d-41a6-9821-4bd734cb14ad admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.399356 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd521e-ecfe-4b68-b1ef-ca066a336f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.400214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd521e-ecfe-4b68-b1ef-ca066a336f04 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.400600 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd521e-ecfe-4b68-b1ef-ca066a336f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.400979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd521e-ecfe-4b68-b1ef-ca066a336f04 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.401892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd521e-ecfe-4b68-b1ef-ca066a336f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.404690 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd521e-ecfe-4b68-b1ef-ca066a336f04 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.494732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1346/2709] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:31 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:33:31.498557 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ce1db-994d-41a6-9821-4bd734cb14ad admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.508131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ce1db-994d-41a6-9821-4bd734cb14ad admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.512891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1ef48-ab7b-4ccd-a565-2804f0031684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.513477 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1ef48-ab7b-4ccd-a565-2804f0031684 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.513757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1ef48-ab7b-4ccd-a565-2804f0031684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.514033 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1ef48-ab7b-4ccd-a565-2804f0031684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.514695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1ef48-ab7b-4ccd-a565-2804f0031684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.516328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1ef48-ab7b-4ccd-a565-2804f0031684 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.523319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1364/2710] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:31 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:33:31.526821 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1347/2711] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:33:31.527483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd521e-ecfe-4b68-b1ef-ca066a336f04 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.530470 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1ef48-ab7b-4ccd-a565-2804f0031684 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.537608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd521e-ecfe-4b68-b1ef-ca066a336f04 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.541027 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1ef48-ab7b-4ccd-a565-2804f0031684 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.542164 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8eac9-0d5b-4f0e-b60c-73c5610766dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.542760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8eac9-0d5b-4f0e-b60c-73c5610766dc admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.543039 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8eac9-0d5b-4f0e-b60c-73c5610766dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.543310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8eac9-0d5b-4f0e-b60c-73c5610766dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.543972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8eac9-0d5b-4f0e-b60c-73c5610766dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.545561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8eac9-0d5b-4f0e-b60c-73c5610766dc admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.545797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645e293-90ec-4fc1-9b88-142c2c4e03b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.546383 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645e293-90ec-4fc1-9b88-142c2c4e03b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.546805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645e293-90ec-4fc1-9b88-142c2c4e03b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.546964 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9645e293-90ec-4fc1-9b88-142c2c4e03b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.547595 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645e293-90ec-4fc1-9b88-142c2c4e03b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.549909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9645e293-90ec-4fc1-9b88-142c2c4e03b5 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.556216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1365/2712] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:33:31.560268 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8eac9-0d5b-4f0e-b60c-73c5610766dc admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.560993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1348/2713] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:33:31.564790 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645e293-90ec-4fc1-9b88-142c2c4e03b5 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.570921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8eac9-0d5b-4f0e-b60c-73c5610766dc admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.575842 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9645e293-90ec-4fc1-9b88-142c2c4e03b5 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.575994 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18117b3-3e65-4815-aa3d-fd526f85e1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.576305 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18117b3-3e65-4815-aa3d-fd526f85e1fc admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.576582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18117b3-3e65-4815-aa3d-fd526f85e1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.576975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18117b3-3e65-4815-aa3d-fd526f85e1fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.577617 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18117b3-3e65-4815-aa3d-fd526f85e1fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.579219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18117b3-3e65-4815-aa3d-fd526f85e1fc admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.580274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a9774-6e36-4c58-bea1-b66130b1ad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.580979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9774-6e36-4c58-bea1-b66130b1ad4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.581255 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9774-6e36-4c58-bea1-b66130b1ad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.581535 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a9774-6e36-4c58-bea1-b66130b1ad4f admin admin] PATH_INFO: `/v3/projects/7b970dd3eba944d082e74ffa26989db9/users/115054a983d646d78cfc13a1e577f48c/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.591063 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1366/2714] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:31.595879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18117b3-3e65-4815-aa3d-fd526f85e1fc admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.597194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a9774-6e36-4c58-bea1-b66130b1ad4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b970dd3eba944d082e74ffa26989db9, user_id=115054a983d646d78cfc13a1e577f48c, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.599891 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a9774-6e36-4c58-bea1-b66130b1ad4f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.605694 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18117b3-3e65-4815-aa3d-fd526f85e1fc admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.611031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6c78f-9bbe-4da9-8b78-6caba70325f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.611615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6c78f-9bbe-4da9-8b78-6caba70325f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.615169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6c78f-9bbe-4da9-8b78-6caba70325f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.615169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6c78f-9bbe-4da9-8b78-6caba70325f4 admin admin] PATH_INFO: `/v3/projects/3ec6a0ab035148cd9466f28ad3f1ae59/users/b7fdcf3f310b455d925e277ab48828ec/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.626735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6c78f-9bbe-4da9-8b78-6caba70325f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec6a0ab035148cd9466f28ad3f1ae59, user_id=b7fdcf3f310b455d925e277ab48828ec, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.629511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6c78f-9bbe-4da9-8b78-6caba70325f4 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.654481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1349/2715] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:31 2022] PUT /identity/v3/projects/7b970dd3eba944d082e74ffa26989db9/users/115054a983d646d78cfc13a1e577f48c/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:33:31.665379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f0a68-e405-44b0-bf2b-088b52184960 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.665666 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f0a68-e405-44b0-bf2b-088b52184960 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.665947 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f0a68-e405-44b0-bf2b-088b52184960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.692532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:31.696378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-267f0a68-e405-44b0-bf2b-088b52184960 None None] MFA Rules not processed for user `115054a983d646d78cfc13a1e577f48c`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:31.697839 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-267f0a68-e405-44b0-bf2b-088b52184960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:31.700657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1367/2716] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:31 2022] PUT /identity/v3/projects/3ec6a0ab035148cd9466f28ad3f1ae59/users/b7fdcf3f310b455d925e277ab48828ec/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:33:31.707301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12986b6-ca25-49fc-a147-af54a6168392 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.707301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12986b6-ca25-49fc-a147-af54a6168392 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.707301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12986b6-ca25-49fc-a147-af54a6168392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.727644 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-a12986b6-ca25-49fc-a147-af54a6168392 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:31.729804 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a12986b6-ca25-49fc-a147-af54a6168392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:31.751696 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:31.770685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1350/2717] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:33:31.775866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f0a68-e405-44b0-bf2b-088b52184960 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.792678 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1368/2718] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:33:31.803662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12986b6-ca25-49fc-a147-af54a6168392 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.805100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a12986b6-ca25-49fc-a147-af54a6168392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:31.847816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f0a68-e405-44b0-bf2b-088b52184960 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.852470 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d6649-5ccc-48d8-9dbd-3da28c953ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.853137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d6649-5ccc-48d8-9dbd-3da28c953ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.853560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d6649-5ccc-48d8-9dbd-3da28c953ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.853836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d6649-5ccc-48d8-9dbd-3da28c953ee6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.854465 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d6649-5ccc-48d8-9dbd-3da28c953ee6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.856396 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d6649-5ccc-48d8-9dbd-3da28c953ee6 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.868238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12986b6-ca25-49fc-a147-af54a6168392 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:31.873312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f7a71-6d20-409a-b070-99b8ff1bb3e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:31.874479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f7a71-6d20-409a-b070-99b8ff1bb3e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:31.874479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f7a71-6d20-409a-b070-99b8ff1bb3e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:31.874930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f7a71-6d20-409a-b070-99b8ff1bb3e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:31.887737 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f7a71-6d20-409a-b070-99b8ff1bb3e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:31.889194 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f7a71-6d20-409a-b070-99b8ff1bb3e1 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:31.899111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1369/2719] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:31.943667 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1351/2720] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:31 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:31.947705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f7a71-6d20-409a-b070-99b8ff1bb3e1 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:31.957669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f7a71-6d20-409a-b070-99b8ff1bb3e1 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:32.022549 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebf3f8-8151-4ba8-8610-e437a086fbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:32.023205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebf3f8-8151-4ba8-8610-e437a086fbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:32.023485 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebf3f8-8151-4ba8-8610-e437a086fbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:32.023778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebf3f8-8151-4ba8-8610-e437a086fbb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:32.024460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebf3f8-8151-4ba8-8610-e437a086fbb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:32.026390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ebf3f8-8151-4ba8-8610-e437a086fbb5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:32.111649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1370/2721] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:31 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:33:32.115879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d6649-5ccc-48d8-9dbd-3da28c953ee6 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:32.127492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d6649-5ccc-48d8-9dbd-3da28c953ee6 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:32.132318 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5200b-c1ca-4d09-a23c-7acf32c658af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:32.137649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5200b-c1ca-4d09-a23c-7acf32c658af admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:32.137649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5200b-c1ca-4d09-a23c-7acf32c658af admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:32.137649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa5200b-c1ca-4d09-a23c-7acf32c658af admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:32.137649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5200b-c1ca-4d09-a23c-7acf32c658af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:32.137649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa5200b-c1ca-4d09-a23c-7acf32c658af admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:32.147391 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1352/2722] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:32.151568 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebf3f8-8151-4ba8-8610-e437a086fbb5 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:32.161671 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebf3f8-8151-4ba8-8610-e437a086fbb5 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:32.166675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f251e-9577-4099-bc92-c577c0e1b988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:32.167328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f251e-9577-4099-bc92-c577c0e1b988 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:32.167605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f251e-9577-4099-bc92-c577c0e1b988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:32.167906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f251e-9577-4099-bc92-c577c0e1b988 admin admin] PATH_INFO: `/v3/projects/37f9c963187d4d3fa7a43e2d00ed65b6/users/7dc5f89ee8cc468c9baae00f7ca5ae20/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:32.182860 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f251e-9577-4099-bc92-c577c0e1b988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f9c963187d4d3fa7a43e2d00ed65b6, user_id=7dc5f89ee8cc468c9baae00f7ca5ae20, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:32.185604 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f251e-9577-4099-bc92-c577c0e1b988 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:32.272635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1371/2723] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:32 2022] PUT /identity/v3/projects/37f9c963187d4d3fa7a43e2d00ed65b6/users/7dc5f89ee8cc468c9baae00f7ca5ae20/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:32.276644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5200b-c1ca-4d09-a23c-7acf32c658af admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:32.286636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa5200b-c1ca-4d09-a23c-7acf32c658af admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:32.344099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8c76a-9b1a-437e-900f-354a53bd71c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:32.344712 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8c76a-9b1a-437e-900f-354a53bd71c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:32.345063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8c76a-9b1a-437e-900f-354a53bd71c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:32.345296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8c76a-9b1a-437e-900f-354a53bd71c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:32.345946 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8c76a-9b1a-437e-900f-354a53bd71c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:32.347539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8c76a-9b1a-437e-900f-354a53bd71c5 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:32.358077 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1353/2724] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:32.362561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f251e-9577-4099-bc92-c577c0e1b988 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:32.373319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f251e-9577-4099-bc92-c577c0e1b988 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:32.378518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f52c16-dae0-4cf8-9e9a-aa291c9d890a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:32.379167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f52c16-dae0-4cf8-9e9a-aa291c9d890a admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:32.379440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f52c16-dae0-4cf8-9e9a-aa291c9d890a admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:32.379728 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f52c16-dae0-4cf8-9e9a-aa291c9d890a admin admin] PATH_INFO: `/v3/projects/37f9c963187d4d3fa7a43e2d00ed65b6/users/7dc5f89ee8cc468c9baae00f7ca5ae20/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:32.381638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f52c16-dae0-4cf8-9e9a-aa291c9d890a admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f9c963187d4d3fa7a43e2d00ed65b6, user_id=7dc5f89ee8cc468c9baae00f7ca5ae20, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:32.384578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f52c16-dae0-4cf8-9e9a-aa291c9d890a admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:32.401426 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1372/2725] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:32 2022] PUT /identity/v3/projects/37f9c963187d4d3fa7a43e2d00ed65b6/users/7dc5f89ee8cc468c9baae00f7ca5ae20/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:32.413941 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580625-797e-476a-ac42-e5669086e094 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:32.414339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580625-797e-476a-ac42-e5669086e094 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:32.414520 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31580625-797e-476a-ac42-e5669086e094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:32.447538 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:32.451692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-31580625-797e-476a-ac42-e5669086e094 None None] MFA Rules not processed for user `7dc5f89ee8cc468c9baae00f7ca5ae20`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:32.453727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-31580625-797e-476a-ac42-e5669086e094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:32.538526 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1354/2726] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:32.550573 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f52c16-dae0-4cf8-9e9a-aa291c9d890a admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:32.552706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-73f52c16-dae0-4cf8-9e9a-aa291c9d890a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:32.580601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:32.613022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f52c16-dae0-4cf8-9e9a-aa291c9d890a admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:32.616878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc65bee-82db-45fe-ace7-4c6907f78583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:32.617507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc65bee-82db-45fe-ace7-4c6907f78583 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:32.617785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc65bee-82db-45fe-ace7-4c6907f78583 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:32.618057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc65bee-82db-45fe-ace7-4c6907f78583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:32.673931 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc65bee-82db-45fe-ace7-4c6907f78583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:32.675393 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc65bee-82db-45fe-ace7-4c6907f78583 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:32.684253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1373/2727] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:34.736213 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31580625-797e-476a-ac42-e5669086e094 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:34.738460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-31580625-797e-476a-ac42-e5669086e094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:34.755572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31580625-797e-476a-ac42-e5669086e094 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:34.759584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f32e8-4b91-4bbb-b04c-c9cde5e17f1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:34.762107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f32e8-4b91-4bbb-b04c-c9cde5e17f1e service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:34.762107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f32e8-4b91-4bbb-b04c-c9cde5e17f1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:34.762107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f32e8-4b91-4bbb-b04c-c9cde5e17f1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:34.824569 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f32e8-4b91-4bbb-b04c-c9cde5e17f1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:34.826587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f32e8-4b91-4bbb-b04c-c9cde5e17f1e service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:34.838303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1355/2728] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:35.336759 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc65bee-82db-45fe-ace7-4c6907f78583 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:35.338235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2bc65bee-82db-45fe-ace7-4c6907f78583 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:35.405127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc65bee-82db-45fe-ace7-4c6907f78583 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:35.408995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed13b8-0ef0-44bb-ba31-4d0734999ba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:35.409638 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed13b8-0ef0-44bb-ba31-4d0734999ba1 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:35.409928 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed13b8-0ef0-44bb-ba31-4d0734999ba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:35.410197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed13b8-0ef0-44bb-ba31-4d0734999ba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:35.411965 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-94ed13b8-0ef0-44bb-ba31-4d0734999ba1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:35.424028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed13b8-0ef0-44bb-ba31-4d0734999ba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:35.424389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ed13b8-0ef0-44bb-ba31-4d0734999ba1 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:35.433389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1374/2729] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:36.485589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f32e8-4b91-4bbb-b04c-c9cde5e17f1e service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:36.496243 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f32e8-4b91-4bbb-b04c-c9cde5e17f1e service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:36.500178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95531e2-63a0-4d50-ba02-861c9dd1fd3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:36.505801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95531e2-63a0-4d50-ba02-861c9dd1fd3d service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:36.505801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95531e2-63a0-4d50-ba02-861c9dd1fd3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:36.505801 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95531e2-63a0-4d50-ba02-861c9dd1fd3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:36.511189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95531e2-63a0-4d50-ba02-861c9dd1fd3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:36.512722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95531e2-63a0-4d50-ba02-861c9dd1fd3d service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:36.523344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1356/2730] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:37.319434 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df109-daa2-4558-8646-e731975caf1a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:37.320017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df109-daa2-4558-8646-e731975caf1a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:37.320017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df109-daa2-4558-8646-e731975caf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:37.349588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-2f5df109-daa2-4558-8646-e731975caf1a None None] MFA Rules not processed for user `b7fdcf3f310b455d925e277ab48828ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:37.351193 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2f5df109-daa2-4558-8646-e731975caf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:37.432517 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1375/2731] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:37.437132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95531e2-63a0-4d50-ba02-861c9dd1fd3d service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:37.504858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95531e2-63a0-4d50-ba02-861c9dd1fd3d service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:37.509725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d0008-5e54-458d-ab04-990d8ddb56fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:37.510333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0008-5e54-458d-ab04-990d8ddb56fc admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:37.510614 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0008-5e54-458d-ab04-990d8ddb56fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:37.510890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0008-5e54-458d-ab04-990d8ddb56fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:37.511537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d0008-5e54-458d-ab04-990d8ddb56fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:37.513564 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d0008-5e54-458d-ab04-990d8ddb56fc admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:37.591180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1357/2732] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:37 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:37.592421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5df109-daa2-4558-8646-e731975caf1a None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:37.604354 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5df109-daa2-4558-8646-e731975caf1a None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:37.609809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:37.665327 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b26634-cb7a-4338-bb8f-f2caae6ff5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:37.666081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b26634-cb7a-4338-bb8f-f2caae6ff5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:37.666449 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b26634-cb7a-4338-bb8f-f2caae6ff5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:37.666621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b26634-cb7a-4338-bb8f-f2caae6ff5d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:37.667257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b26634-cb7a-4338-bb8f-f2caae6ff5d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:37.669214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b26634-cb7a-4338-bb8f-f2caae6ff5d2 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:37.757451 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1376/2733] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:37 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:37.761574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d0008-5e54-458d-ab04-990d8ddb56fc admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:37.773210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d0008-5e54-458d-ab04-990d8ddb56fc admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:37.778320 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd7f31-eccd-47f4-8a12-6ae5e4562d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:37.779171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd7f31-eccd-47f4-8a12-6ae5e4562d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:37.779640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd7f31-eccd-47f4-8a12-6ae5e4562d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:37.780618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd7f31-eccd-47f4-8a12-6ae5e4562d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:37.781327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd7f31-eccd-47f4-8a12-6ae5e4562d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:37.783599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd7f31-eccd-47f4-8a12-6ae5e4562d7f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:37.786103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:37.795264 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1358/2734] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:37.799614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b26634-cb7a-4338-bb8f-f2caae6ff5d2 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:37.819322 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b26634-cb7a-4338-bb8f-f2caae6ff5d2 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:37.821419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452eb98-9ecd-40a0-bba4-b30ebde2e4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:37.822101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452eb98-9ecd-40a0-bba4-b30ebde2e4c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:37.822377 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452eb98-9ecd-40a0-bba4-b30ebde2e4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:37.822652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452eb98-9ecd-40a0-bba4-b30ebde2e4c0 admin admin] PATH_INFO: `/v3/projects/6a79d8aa0fd847cd8d7fc19357784ed0/users/0769a62c8f644d93881d4fa794931fa8/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:37.842320 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452eb98-9ecd-40a0-bba4-b30ebde2e4c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a79d8aa0fd847cd8d7fc19357784ed0, user_id=0769a62c8f644d93881d4fa794931fa8, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:37.845645 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4452eb98-9ecd-40a0-bba4-b30ebde2e4c0 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:37.927728 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1377/2735] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:37 2022] PUT /identity/v3/projects/6a79d8aa0fd847cd8d7fc19357784ed0/users/0769a62c8f644d93881d4fa794931fa8/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:37.934797 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd7f31-eccd-47f4-8a12-6ae5e4562d7f admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:37.946546 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd7f31-eccd-47f4-8a12-6ae5e4562d7f admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:38.033841 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc5dc9-15a1-444f-94f8-ad0482feee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:38.035192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc5dc9-15a1-444f-94f8-ad0482feee5f admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:38.035192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc5dc9-15a1-444f-94f8-ad0482feee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:38.035192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfc5dc9-15a1-444f-94f8-ad0482feee5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:38.035657 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc5dc9-15a1-444f-94f8-ad0482feee5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:38.037299 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfc5dc9-15a1-444f-94f8-ad0482feee5f admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:38.048331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1359/2736] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:38.052708 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452eb98-9ecd-40a0-bba4-b30ebde2e4c0 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:38.063274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452eb98-9ecd-40a0-bba4-b30ebde2e4c0 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:38.068643 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd2e42-b37b-490e-b078-5517f4c82cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:38.069319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd2e42-b37b-490e-b078-5517f4c82cd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:38.069594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd2e42-b37b-490e-b078-5517f4c82cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:38.069884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd2e42-b37b-490e-b078-5517f4c82cd4 admin admin] PATH_INFO: `/v3/projects/6a79d8aa0fd847cd8d7fc19357784ed0/users/0769a62c8f644d93881d4fa794931fa8/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:38.072042 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd2e42-b37b-490e-b078-5517f4c82cd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a79d8aa0fd847cd8d7fc19357784ed0, user_id=0769a62c8f644d93881d4fa794931fa8, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:38.074978 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd2e42-b37b-490e-b078-5517f4c82cd4 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:38.092177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1378/2737] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:38 2022] PUT /identity/v3/projects/6a79d8aa0fd847cd8d7fc19357784ed0/users/0769a62c8f644d93881d4fa794931fa8/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:41.375587 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc5dc9-15a1-444f-94f8-ad0482feee5f admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:41.387366 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfc5dc9-15a1-444f-94f8-ad0482feee5f admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:41.470493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:41.471596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:41.472149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:41.472659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] PATH_INFO: `/v3/users/a269667f143243f5a60bf4cbb3096b09` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:41.474851 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a269667f143243f5a60bf4cbb3096b09)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:41.478730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:41.801531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a269667f143243f5a60bf4cbb3096b09'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:41.901483 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a269667f143243f5a60bf4cbb3096b09'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:41.902136 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a269667f143243f5a60bf4cbb3096b09'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:41.902800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a269667f143243f5a60bf4cbb3096b09'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:41.916927 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a269667f143243f5a60bf4cbb3096b09'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:41.944836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1360/2738] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:33:41 2022] DELETE /identity/v3/users/a269667f143243f5a60bf4cbb3096b09 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:41.945017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d206e9-d63f-4c83-9d00-efd509580531 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:41.945017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d206e9-d63f-4c83-9d00-efd509580531 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:41.945017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d206e9-d63f-4c83-9d00-efd509580531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:41.964657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-e4d206e9-d63f-4c83-9d00-efd509580531 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:41.967010 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e4d206e9-d63f-4c83-9d00-efd509580531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:42.062519 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1379/2739] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:42.069919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:42.071494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:42.089155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:42.152912 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe7ada2-f240-4874-923c-cb2a4a469996 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:42.158021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad47e1-644f-4849-97fb-6b48c7114a4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:42.158648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad47e1-644f-4849-97fb-6b48c7114a4b service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:42.158929 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad47e1-644f-4849-97fb-6b48c7114a4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:42.159244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad47e1-644f-4849-97fb-6b48c7114a4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:42.171911 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad47e1-644f-4849-97fb-6b48c7114a4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:42.173898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad47e1-644f-4849-97fb-6b48c7114a4b service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:42.185472 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1361/2740] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:42.607346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d206e9-d63f-4c83-9d00-efd509580531 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:42.607346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e4d206e9-d63f-4c83-9d00-efd509580531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:42.622185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d206e9-d63f-4c83-9d00-efd509580531 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:42.629294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:42.630071 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:42.630826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:42.631139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] PATH_INFO: `/v3/projects/04b40d23e6fe4be6970569b115034bc7` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:42.632892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] RBAC: Authorizing `identity:delete_project(project_id=04b40d23e6fe4be6970569b115034bc7)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:42.642154 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:42.810903 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04b40d23e6fe4be6970569b115034bc7', 'request_id': 'req-f74094b4-a92b-4960-a232-781ba869821e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:42.818615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:42.820570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04b40d23e6fe4be6970569b115034bc7', 'request_id': 'req-f74094b4-a92b-4960-a232-781ba869821e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:42.913927 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04b40d23e6fe4be6970569b115034bc7', 'request_id': 'req-f74094b4-a92b-4960-a232-781ba869821e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:42.926856 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] The token cache is being invalidate because project 04b40d23e6fe4be6970569b115034bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:33:42.927225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:42.927918 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f74094b4-a92b-4960-a232-781ba869821e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:42.929671 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1380/2741] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:33:42 2022] DELETE /identity/v3/projects/04b40d23e6fe4be6970569b115034bc7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:42.933668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad47e1-644f-4849-97fb-6b48c7114a4b service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:42.945495 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad47e1-644f-4849-97fb-6b48c7114a4b service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:43.039309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:43.040470 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:43.040777 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:43.041026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] PATH_INFO: `/v3/users/115054a983d646d78cfc13a1e577f48c` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:43.042403 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=115054a983d646d78cfc13a1e577f48c)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:43.044677 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:43.223873 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '115054a983d646d78cfc13a1e577f48c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:43.317576 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '115054a983d646d78cfc13a1e577f48c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:43.318282 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '115054a983d646d78cfc13a1e577f48c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:43.318951 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '115054a983d646d78cfc13a1e577f48c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:43.327095 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '115054a983d646d78cfc13a1e577f48c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:43.350770 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1362/2742] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:33:42 2022] DELETE /identity/v3/users/115054a983d646d78cfc13a1e577f48c => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:43.813583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d55ba-a669-4db8-a3b0-801b215c52bf None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:43.813583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d55ba-a669-4db8-a3b0-801b215c52bf None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:43.813583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d55ba-a669-4db8-a3b0-801b215c52bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:43.843053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-128d55ba-a669-4db8-a3b0-801b215c52bf None None] MFA Rules not processed for user `0769a62c8f644d93881d4fa794931fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:43.843053 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-128d55ba-a669-4db8-a3b0-801b215c52bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:43.875676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:43.881144 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:43.970915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:43.987074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1381/2743] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:44.004013 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d55ba-a669-4db8-a3b0-801b215c52bf None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:44.006332 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-128d55ba-a669-4db8-a3b0-801b215c52bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:44.020417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e287e0-3307-4d3f-9e26-84140ce59540 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:44.034745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:44.035709 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:44.036082 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:44.036368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] PATH_INFO: `/v3/projects/7b970dd3eba944d082e74ffa26989db9` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:44.037986 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b970dd3eba944d082e74ffa26989db9)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:44.040985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:44.092518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d55ba-a669-4db8-a3b0-801b215c52bf None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:44.097235 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec721b0-9797-42f9-bf72-5c14d61bb006 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5d7cf7534941cb857a4ab5867d978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:44.097861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec721b0-9797-42f9-bf72-5c14d61bb006 service glance] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:44.098141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec721b0-9797-42f9-bf72-5c14d61bb006 service glance] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:44.098415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec721b0-9797-42f9-bf72-5c14d61bb006 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:44.100270 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fec721b0-9797-42f9-bf72-5c14d61bb006 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:44.159806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b970dd3eba944d082e74ffa26989db9', 'request_id': 'req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:44.167583 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec721b0-9797-42f9-bf72-5c14d61bb006 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:44.170120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec721b0-9797-42f9-bf72-5c14d61bb006 service glance] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:44.180112 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1382/2744] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:33:44.242551 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b970dd3eba944d082e74ffa26989db9', 'request_id': 'req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:44.250594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b970dd3eba944d082e74ffa26989db9', 'request_id': 'req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:44.262329 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] The token cache is being invalidate because project 7b970dd3eba944d082e74ffa26989db9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:33:44.262618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:44.263274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9a3c4c9e-dec5-4520-b840-de10d7b3b1fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:44.265218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1363/2745] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:33:43 2022] DELETE /identity/v3/projects/7b970dd3eba944d082e74ffa26989db9 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:33:44.269502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec721b0-9797-42f9-bf72-5c14d61bb006 service glance] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:44.280855 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec721b0-9797-42f9-bf72-5c14d61bb006 service glance] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:44.301421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:44.352974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:44.353625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:44.353904 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:44.354178 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] PATH_INFO: `/v3/users/7dc5f89ee8cc468c9baae00f7ca5ae20` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:44.355398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc5f89ee8cc468c9baae00f7ca5ae20)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:44.357376 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:44.486115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adede1c-e9a6-4dbc-9b8f-e5999529b455 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:44.486469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adede1c-e9a6-4dbc-9b8f-e5999529b455 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:44.486753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adede1c-e9a6-4dbc-9b8f-e5999529b455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:44.545735 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-3adede1c-e9a6-4dbc-9b8f-e5999529b455 None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:44.546807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3adede1c-e9a6-4dbc-9b8f-e5999529b455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:44.567998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc5f89ee8cc468c9baae00f7ca5ae20'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:44.618555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1364/2746] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:33:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:33:44.643697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc5f89ee8cc468c9baae00f7ca5ae20'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:44.644397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc5f89ee8cc468c9baae00f7ca5ae20'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:44.645210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc5f89ee8cc468c9baae00f7ca5ae20'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:44.654533 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc5f89ee8cc468c9baae00f7ca5ae20'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:44.673220 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1383/2747] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:33:44 2022] DELETE /identity/v3/users/7dc5f89ee8cc468c9baae00f7ca5ae20 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:44.768195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adede1c-e9a6-4dbc-9b8f-e5999529b455 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:44.769704 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3adede1c-e9a6-4dbc-9b8f-e5999529b455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:44.817620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:44.873618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adede1c-e9a6-4dbc-9b8f-e5999529b455 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:44.951783 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b117ee7-70fd-4a30-902f-2d91fe186bfc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:44.952507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b117ee7-70fd-4a30-902f-2d91fe186bfc service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:44.952793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b117ee7-70fd-4a30-902f-2d91fe186bfc service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:44.953081 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b117ee7-70fd-4a30-902f-2d91fe186bfc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:44.954996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-0b117ee7-70fd-4a30-902f-2d91fe186bfc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:45.028157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b117ee7-70fd-4a30-902f-2d91fe186bfc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:45.029770 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b117ee7-70fd-4a30-902f-2d91fe186bfc service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:45.039461 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1365/2748] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:33:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 16:33:45.075942 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:45.078019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:45.182410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d428d-4603-40f1-8f47-e1e957ee7245 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:45.187535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:45.188334 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:45.188697 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:45.189041 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] PATH_INFO: `/v3/projects/37f9c963187d4d3fa7a43e2d00ed65b6` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:45.190236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f9c963187d4d3fa7a43e2d00ed65b6)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:45.192734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:45.317308 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:45.321539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f9c963187d4d3fa7a43e2d00ed65b6', 'request_id': 'req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:45.327332 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f9c963187d4d3fa7a43e2d00ed65b6', 'request_id': 'req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:45.409606 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f9c963187d4d3fa7a43e2d00ed65b6', 'request_id': 'req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:45.422659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] The token cache is being invalidate because project 37f9c963187d4d3fa7a43e2d00ed65b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:33:45.423092 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:45.423861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6c96a05a-a3fd-4424-b804-f5c1e9bbc269 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:33:45.425588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1384/2749] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:33:45 2022] DELETE /identity/v3/projects/37f9c963187d4d3fa7a43e2d00ed65b6 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:45.430544 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad609297-9088-479a-bd6c-2e330111bd5d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:45.430916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad609297-9088-479a-bd6c-2e330111bd5d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:45.431200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad609297-9088-479a-bd6c-2e330111bd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:45.458739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ad609297-9088-479a-bd6c-2e330111bd5d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:45.458739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ad609297-9088-479a-bd6c-2e330111bd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:45.561375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1366/2750] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:45.571499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84022e0-339e-4698-9bc7-226c38c114e4 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:45.572129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84022e0-339e-4698-9bc7-226c38c114e4 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:45.572489 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84022e0-339e-4698-9bc7-226c38c114e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:45.598379 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-f84022e0-339e-4698-9bc7-226c38c114e4 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:45.599751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f84022e0-339e-4698-9bc7-226c38c114e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:45.610106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1385/2751] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:45.614379 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad609297-9088-479a-bd6c-2e330111bd5d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:45.619407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:45.628942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad609297-9088-479a-bd6c-2e330111bd5d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:45.644155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1ab37-1ba0-4d0e-a016-7f11c8e5a223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:45.644784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1ab37-1ba0-4d0e-a016-7f11c8e5a223 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:45.645060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1ab37-1ba0-4d0e-a016-7f11c8e5a223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:45.645331 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1ab37-1ba0-4d0e-a016-7f11c8e5a223 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:45.646047 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1ab37-1ba0-4d0e-a016-7f11c8e5a223 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:33:45.646309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1ab37-1ba0-4d0e-a016-7f11c8e5a223 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:45.647993 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1ab37-1ba0-4d0e-a016-7f11c8e5a223 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:45.670676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1367/2752] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:33:45.674650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84022e0-339e-4698-9bc7-226c38c114e4 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:45.694832 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84022e0-339e-4698-9bc7-226c38c114e4 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:45.700756 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069c2bd-9ef7-416b-8501-17f78c081eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:45.701445 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069c2bd-9ef7-416b-8501-17f78c081eca admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:45.701721 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069c2bd-9ef7-416b-8501-17f78c081eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:45.702000 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069c2bd-9ef7-416b-8501-17f78c081eca admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:45.702640 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069c2bd-9ef7-416b-8501-17f78c081eca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:45.705346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069c2bd-9ef7-416b-8501-17f78c081eca admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:45.791322 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1386/2753] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:33:45 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:45.795281 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1ab37-1ba0-4d0e-a016-7f11c8e5a223 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:45.806847 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1ab37-1ba0-4d0e-a016-7f11c8e5a223 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:45.889159 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d830598-406d-4a7f-9639-80c69005e862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:45.890055 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d830598-406d-4a7f-9639-80c69005e862 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:45.890490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d830598-406d-4a7f-9639-80c69005e862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:45.892747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d830598-406d-4a7f-9639-80c69005e862 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:45.896277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d830598-406d-4a7f-9639-80c69005e862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:45.898757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d830598-406d-4a7f-9639-80c69005e862 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:45.973518 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069c2bd-9ef7-416b-8501-17f78c081eca admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:45.974123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1368/2754] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:33:45 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:33:45.989219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069c2bd-9ef7-416b-8501-17f78c081eca admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:45.999172 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726b39b-0479-4f8d-8308-3b00bb4414bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:45.999977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726b39b-0479-4f8d-8308-3b00bb4414bc admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:46.000547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726b39b-0479-4f8d-8308-3b00bb4414bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:46.001488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726b39b-0479-4f8d-8308-3b00bb4414bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:46.002952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726b39b-0479-4f8d-8308-3b00bb4414bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:46.008035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726b39b-0479-4f8d-8308-3b00bb4414bc admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:46.032407 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1387/2755] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:46.035444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d830598-406d-4a7f-9639-80c69005e862 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:46.046113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d830598-406d-4a7f-9639-80c69005e862 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:46.052062 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e12066-aeaf-4258-86d5-aa14e8d2eb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:46.053233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e12066-aeaf-4258-86d5-aa14e8d2eb42 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:46.053523 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e12066-aeaf-4258-86d5-aa14e8d2eb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:46.053812 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e12066-aeaf-4258-86d5-aa14e8d2eb42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:46.054460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e12066-aeaf-4258-86d5-aa14e8d2eb42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:46.056823 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e12066-aeaf-4258-86d5-aa14e8d2eb42 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:46.076818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1369/2756] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:33:46.081751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726b39b-0479-4f8d-8308-3b00bb4414bc admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:46.098403 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726b39b-0479-4f8d-8308-3b00bb4414bc admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:46.103955 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58289c-69bd-413d-8696-961bb1132ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:46.104990 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58289c-69bd-413d-8696-961bb1132ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:46.105468 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58289c-69bd-413d-8696-961bb1132ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:46.105822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58289c-69bd-413d-8696-961bb1132ddd admin admin] PATH_INFO: `/v3/projects/ec508512f403428a8b5f6a542cf14ced/users/5424717a9db84dd68ba36eb6db7b69f2/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:46.123458 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58289c-69bd-413d-8696-961bb1132ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec508512f403428a8b5f6a542cf14ced, user_id=5424717a9db84dd68ba36eb6db7b69f2, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:46.127225 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58289c-69bd-413d-8696-961bb1132ddd admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:46.209545 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1388/2757] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:33:46 2022] PUT /identity/v3/projects/ec508512f403428a8b5f6a542cf14ced/users/5424717a9db84dd68ba36eb6db7b69f2/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:33:46.289301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39307-cd3e-49a4-8fe7-e97af8563a9c None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:46.289582 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39307-cd3e-49a4-8fe7-e97af8563a9c None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:46.289857 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd39307-cd3e-49a4-8fe7-e97af8563a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:46.337702 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-8fd39307-cd3e-49a4-8fe7-e97af8563a9c None None] MFA Rules not processed for user `5424717a9db84dd68ba36eb6db7b69f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:46.339036 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8fd39307-cd3e-49a4-8fe7-e97af8563a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:46.428612 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1370/2758] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:46.440097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58289c-69bd-413d-8696-961bb1132ddd admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:46.444699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0b58289c-69bd-413d-8696-961bb1132ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:46.524219 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58289c-69bd-413d-8696-961bb1132ddd admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:46.529121 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5c42b-7a7f-4bb8-9699-9a9c4aaf8873 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a5d7cf7534941cb857a4ab5867d978e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:46.529753 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c42b-7a7f-4bb8-9699-9a9c4aaf8873 service glance] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:46.530028 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c42b-7a7f-4bb8-9699-9a9c4aaf8873 service glance] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:46.530448 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5c42b-7a7f-4bb8-9699-9a9c4aaf8873 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:46.544318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5c42b-7a7f-4bb8-9699-9a9c4aaf8873 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:46.545688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5c42b-7a7f-4bb8-9699-9a9c4aaf8873 service glance] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:46.564772 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1389/2759] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:55.568671 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb14d6f-6b41-4b89-bdf6-c7682e34c190 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:55.570063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb14d6f-6b41-4b89-bdf6-c7682e34c190 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:55.570063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb14d6f-6b41-4b89-bdf6-c7682e34c190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:55.588860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-1cb14d6f-6b41-4b89-bdf6-c7682e34c190 None None] MFA Rules not processed for user `b7fdcf3f310b455d925e277ab48828ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:55.591270 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1cb14d6f-6b41-4b89-bdf6-c7682e34c190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:55.646897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1371/2760] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:33:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:33:55.903211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5c42b-7a7f-4bb8-9699-9a9c4aaf8873 service glance] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:55.904629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c4d5c42b-7a7f-4bb8-9699-9a9c4aaf8873 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:55.962751 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5c42b-7a7f-4bb8-9699-9a9c4aaf8873 service glance] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:55.966334 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879143d-1335-4d7b-8165-e884e47588a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:55.966934 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879143d-1335-4d7b-8165-e884e47588a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:55.967213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879143d-1335-4d7b-8165-e884e47588a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:55.967490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879143d-1335-4d7b-8165-e884e47588a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:55.969239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2879143d-1335-4d7b-8165-e884e47588a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:55.978539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879143d-1335-4d7b-8165-e884e47588a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:55.979969 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879143d-1335-4d7b-8165-e884e47588a6 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:55.988781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1390/2761] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:56.406099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb14d6f-6b41-4b89-bdf6-c7682e34c190 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:56.408086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1cb14d6f-6b41-4b89-bdf6-c7682e34c190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:56.413336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:56.480941 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb14d6f-6b41-4b89-bdf6-c7682e34c190 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:56.486826 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be792f-59bb-4d2e-958f-5dbc3fc8735d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:56.487475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be792f-59bb-4d2e-958f-5dbc3fc8735d service placement] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:56.487757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be792f-59bb-4d2e-958f-5dbc3fc8735d service placement] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:56.488059 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be792f-59bb-4d2e-958f-5dbc3fc8735d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:56.489793 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-46be792f-59bb-4d2e-958f-5dbc3fc8735d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:56.501957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be792f-59bb-4d2e-958f-5dbc3fc8735d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:56.503523 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be792f-59bb-4d2e-958f-5dbc3fc8735d service placement] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:56.515879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1372/2762] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:33:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:33:59.320614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eef7c-a228-4fa3-aba1-d4f8348a5c35 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:59.324118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eef7c-a228-4fa3-aba1-d4f8348a5c35 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:59.324118 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eef7c-a228-4fa3-aba1-d4f8348a5c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:59.340574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8d0eef7c-a228-4fa3-aba1-d4f8348a5c35 None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:33:59.341774 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8d0eef7c-a228-4fa3-aba1-d4f8348a5c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:59.351614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:33:59.396526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1391/2763] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:33:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:33:59.411485 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be792f-59bb-4d2e-958f-5dbc3fc8735d service placement] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:33:59.413072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-46be792f-59bb-4d2e-958f-5dbc3fc8735d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:33:59.485339 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be792f-59bb-4d2e-958f-5dbc3fc8735d service placement] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:33:59.490512 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899e7cb-3171-42c9-bef4-ba05aa462448 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:33:59.491154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899e7cb-3171-42c9-bef4-ba05aa462448 service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:33:59.491433 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899e7cb-3171-42c9-bef4-ba05aa462448 service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:33:59.491708 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899e7cb-3171-42c9-bef4-ba05aa462448 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:33:59.500104 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899e7cb-3171-42c9-bef4-ba05aa462448 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:33:59.501661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899e7cb-3171-42c9-bef4-ba05aa462448 service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:33:59.510151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1373/2764] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:33:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:34:09.488171 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=18 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:34:09.507086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=63 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:34:16.592274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0eef7c-a228-4fa3-aba1-d4f8348a5c35 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:16.593825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8d0eef7c-a228-4fa3-aba1-d4f8348a5c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:16.678196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0eef7c-a228-4fa3-aba1-d4f8348a5c35 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:16.683834 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f4956-169e-4dcf-bc0a-586d74abdf4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:16.684455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f4956-169e-4dcf-bc0a-586d74abdf4b service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:16.684763 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f4956-169e-4dcf-bc0a-586d74abdf4b service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:16.685011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f4956-169e-4dcf-bc0a-586d74abdf4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:16.686914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f59f4956-169e-4dcf-bc0a-586d74abdf4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:16.700674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f4956-169e-4dcf-bc0a-586d74abdf4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:16.702375 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f4956-169e-4dcf-bc0a-586d74abdf4b service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:16.716632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1392/2765] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:34:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:34:26.240108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899e7cb-3171-42c9-bef4-ba05aa462448 service swift] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:26.241098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b899e7cb-3171-42c9-bef4-ba05aa462448 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:26.256628 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899e7cb-3171-42c9-bef4-ba05aa462448 service swift] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:26.262481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530356e-0c62-4d9b-96bf-09bac8d30cf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:26.263088 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530356e-0c62-4d9b-96bf-09bac8d30cf9 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:26.263365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530356e-0c62-4d9b-96bf-09bac8d30cf9 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:26.263682 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530356e-0c62-4d9b-96bf-09bac8d30cf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:26.265419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2530356e-0c62-4d9b-96bf-09bac8d30cf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:26.277654 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530356e-0c62-4d9b-96bf-09bac8d30cf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:26.279215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530356e-0c62-4d9b-96bf-09bac8d30cf9 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:26.291000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1374/2766] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:34:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:34:26.709591 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=34 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:34:36.286010 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=6 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:34:42.824214 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f4956-169e-4dcf-bc0a-586d74abdf4b service nova] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:42.887130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f4956-169e-4dcf-bc0a-586d74abdf4b service nova] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:42.891614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:42.892276 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:42.893187 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:42.893465 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] PATH_INFO: `/v3/users/5424717a9db84dd68ba36eb6db7b69f2` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:42.894579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5424717a9db84dd68ba36eb6db7b69f2)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:42.896525 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:43.056168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5424717a9db84dd68ba36eb6db7b69f2'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:34:43.117095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5424717a9db84dd68ba36eb6db7b69f2'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:34:43.117754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5424717a9db84dd68ba36eb6db7b69f2'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:34:43.118361 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5424717a9db84dd68ba36eb6db7b69f2'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:34:43.125930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5424717a9db84dd68ba36eb6db7b69f2'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:34:43.143687 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1393/2767] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:34:42 2022] DELETE /identity/v3/users/5424717a9db84dd68ba36eb6db7b69f2 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:34:43.148215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d117e-5e36-4499-92fd-02fc2c6865f6 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:43.150741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d117e-5e36-4499-92fd-02fc2c6865f6 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:43.150741 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d117e-5e36-4499-92fd-02fc2c6865f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:43.167303 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-b10d117e-5e36-4499-92fd-02fc2c6865f6 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:34:43.168541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b10d117e-5e36-4499-92fd-02fc2c6865f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:43.227965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1375/2768] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:34:43.241141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:43.244331 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:43.301091 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e018a-8368-4d22-95e6-1f156ea81b75 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:43.304714 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131622ce-ee16-4ccf-8b9c-56d6bed1b415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:43.305317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131622ce-ee16-4ccf-8b9c-56d6bed1b415 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:43.305597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131622ce-ee16-4ccf-8b9c-56d6bed1b415 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:43.305872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131622ce-ee16-4ccf-8b9c-56d6bed1b415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:43.315186 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131622ce-ee16-4ccf-8b9c-56d6bed1b415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:43.316582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131622ce-ee16-4ccf-8b9c-56d6bed1b415 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:43.326108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1394/2769] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:34:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:34:43.695353 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d117e-5e36-4499-92fd-02fc2c6865f6 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:43.700914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b10d117e-5e36-4499-92fd-02fc2c6865f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:43.709133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d117e-5e36-4499-92fd-02fc2c6865f6 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:43.714296 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:43.715099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:43.715386 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:43.715661 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] PATH_INFO: `/v3/projects/ec508512f403428a8b5f6a542cf14ced` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:43.716874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec508512f403428a8b5f6a542cf14ced)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:43.718838 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:43.820633 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec508512f403428a8b5f6a542cf14ced', 'request_id': 'req-ff0d1fd2-6078-4524-8aff-7fab3914c52e'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:34:43.847643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec508512f403428a8b5f6a542cf14ced', 'request_id': 'req-ff0d1fd2-6078-4524-8aff-7fab3914c52e'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:34:43.853815 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec508512f403428a8b5f6a542cf14ced', 'request_id': 'req-ff0d1fd2-6078-4524-8aff-7fab3914c52e'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:34:43.864539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] The token cache is being invalidate because project ec508512f403428a8b5f6a542cf14ced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:34:43.864825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:34:43.865444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-ff0d1fd2-6078-4524-8aff-7fab3914c52e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:34:43.866909 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1376/2770] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:34:43 2022] DELETE /identity/v3/projects/ec508512f403428a8b5f6a542cf14ced => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:34:43.871717 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e3b74-3110-4a90-a7fd-5853fafd5d48 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:43.872215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e3b74-3110-4a90-a7fd-5853fafd5d48 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:43.872511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1e3b74-3110-4a90-a7fd-5853fafd5d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:43.892366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6e1e3b74-3110-4a90-a7fd-5853fafd5d48 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:34:43.893566 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6e1e3b74-3110-4a90-a7fd-5853fafd5d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:43.909808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:34:43.951636 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1395/2771] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:34:43.959650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4cab5-1aa9-4415-b92a-83c819c52bf7 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:43.959996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4cab5-1aa9-4415-b92a-83c819c52bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:43.960283 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4cab5-1aa9-4415-b92a-83c819c52bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:43.981369 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-92e4cab5-1aa9-4415-b92a-83c819c52bf7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:34:43.982586 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-92e4cab5-1aa9-4415-b92a-83c819c52bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:43.991239 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1377/2772] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:34:43.995483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e3b74-3110-4a90-a7fd-5853fafd5d48 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:44.006310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1e3b74-3110-4a90-a7fd-5853fafd5d48 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:44.012200 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69bf2f-2018-4e9c-927b-9f8b1510ebd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:44.012808 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69bf2f-2018-4e9c-927b-9f8b1510ebd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:44.013087 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69bf2f-2018-4e9c-927b-9f8b1510ebd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:44.013364 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b69bf2f-2018-4e9c-927b-9f8b1510ebd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:44.014019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69bf2f-2018-4e9c-927b-9f8b1510ebd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:34:44.014313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69bf2f-2018-4e9c-927b-9f8b1510ebd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:44.015927 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b69bf2f-2018-4e9c-927b-9f8b1510ebd3 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:44.027684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1396/2773] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:34:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:34:44.032110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4cab5-1aa9-4415-b92a-83c819c52bf7 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:44.041476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4cab5-1aa9-4415-b92a-83c819c52bf7 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:44.046153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f539a-501c-4685-8799-3c066d1bd339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:44.046745 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f539a-501c-4685-8799-3c066d1bd339 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:44.047024 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f539a-501c-4685-8799-3c066d1bd339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:44.047297 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f539a-501c-4685-8799-3c066d1bd339 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:44.047965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f539a-501c-4685-8799-3c066d1bd339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:44.050397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f539a-501c-4685-8799-3c066d1bd339 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:44.112085 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1378/2774] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:34:44 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:34:44.116081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69bf2f-2018-4e9c-927b-9f8b1510ebd3 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:44.126089 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b69bf2f-2018-4e9c-927b-9f8b1510ebd3 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:44.181339 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88331e-135c-4754-a92e-6eecb7986b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:44.181932 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88331e-135c-4754-a92e-6eecb7986b35 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:44.182213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88331e-135c-4754-a92e-6eecb7986b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:44.182487 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88331e-135c-4754-a92e-6eecb7986b35 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:44.183116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88331e-135c-4754-a92e-6eecb7986b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:44.185074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88331e-135c-4754-a92e-6eecb7986b35 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:44.268279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1397/2775] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:34:44 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:34:44.272499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f539a-501c-4685-8799-3c066d1bd339 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:44.280158 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f539a-501c-4685-8799-3c066d1bd339 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:44.285483 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c89e46-6495-4610-bc22-4190f36c3c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:44.286086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c89e46-6495-4610-bc22-4190f36c3c35 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:44.286388 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c89e46-6495-4610-bc22-4190f36c3c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:44.286678 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c89e46-6495-4610-bc22-4190f36c3c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:44.287301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c89e46-6495-4610-bc22-4190f36c3c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:44.288924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c89e46-6495-4610-bc22-4190f36c3c35 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:44.304523 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1379/2776] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:34:44.308480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88331e-135c-4754-a92e-6eecb7986b35 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:44.318748 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88331e-135c-4754-a92e-6eecb7986b35 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:44.323368 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3657258-fb2d-4ca9-a855-8866b7953d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:44.324011 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3657258-fb2d-4ca9-a855-8866b7953d14 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:44.324296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3657258-fb2d-4ca9-a855-8866b7953d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:44.324602 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3657258-fb2d-4ca9-a855-8866b7953d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:44.325234 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3657258-fb2d-4ca9-a855-8866b7953d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:44.326806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3657258-fb2d-4ca9-a855-8866b7953d14 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:44.337168 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1398/2777] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:34:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:34:44.340954 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c89e46-6495-4610-bc22-4190f36c3c35 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:44.351087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c89e46-6495-4610-bc22-4190f36c3c35 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:44.355694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879faab-ba72-4805-ade4-fc07fe5ac3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:44.356362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879faab-ba72-4805-ade4-fc07fe5ac3a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:44.356644 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879faab-ba72-4805-ade4-fc07fe5ac3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:44.356919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e879faab-ba72-4805-ade4-fc07fe5ac3a1 admin admin] PATH_INFO: `/v3/projects/ba0a577cdbb44049934ace0e94520116/users/19092e36ca6d4bc8a576df5e1c7c359d/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:44.371794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879faab-ba72-4805-ade4-fc07fe5ac3a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0a577cdbb44049934ace0e94520116, user_id=19092e36ca6d4bc8a576df5e1c7c359d, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:44.374361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e879faab-ba72-4805-ade4-fc07fe5ac3a1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:44.464919 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1380/2778] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:34:44 2022] PUT /identity/v3/projects/ba0a577cdbb44049934ace0e94520116/users/19092e36ca6d4bc8a576df5e1c7c359d/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:34:44.469976 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3ecc9-aff4-48bf-8291-f3ff884d9448 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:44.470260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3ecc9-aff4-48bf-8291-f3ff884d9448 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:44.470530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3ecc9-aff4-48bf-8291-f3ff884d9448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:44.490166 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-d2d3ecc9-aff4-48bf-8291-f3ff884d9448 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:34:44.491371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d2d3ecc9-aff4-48bf-8291-f3ff884d9448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:44.548201 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1399/2779] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:34:44.557727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879faab-ba72-4805-ade4-fc07fe5ac3a1 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:44.559377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e879faab-ba72-4805-ade4-fc07fe5ac3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:44.581487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:34:44.616730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e879faab-ba72-4805-ade4-fc07fe5ac3a1 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:44.620528 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e423ef9-a8af-4dae-880d-4a2d6b388e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:44.621124 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e423ef9-a8af-4dae-880d-4a2d6b388e3b service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:44.621403 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e423ef9-a8af-4dae-880d-4a2d6b388e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:44.621673 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e423ef9-a8af-4dae-880d-4a2d6b388e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:44.630493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e423ef9-a8af-4dae-880d-4a2d6b388e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:44.631889 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e423ef9-a8af-4dae-880d-4a2d6b388e3b service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:44.641493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1381/2780] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:34:45.752816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114d9f8-96ab-4c5d-b1e3-bf56f3c55330 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:45.753367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114d9f8-96ab-4c5d-b1e3-bf56f3c55330 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:45.753367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114d9f8-96ab-4c5d-b1e3-bf56f3c55330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:45.781294 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-5114d9f8-96ab-4c5d-b1e3-bf56f3c55330 None None] MFA Rules not processed for user `19092e36ca6d4bc8a576df5e1c7c359d`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:34:45.782547 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5114d9f8-96ab-4c5d-b1e3-bf56f3c55330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:45.801572 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:34:45.858246 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1400/2781] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:34:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:34:45.870127 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e423ef9-a8af-4dae-880d-4a2d6b388e3b service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:45.871613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2e423ef9-a8af-4dae-880d-4a2d6b388e3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:45.932007 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e423ef9-a8af-4dae-880d-4a2d6b388e3b service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:45.935669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d653b-0650-4a6e-a72c-696619cf30ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:45.936292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d653b-0650-4a6e-a72c-696619cf30ce service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:45.936571 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d653b-0650-4a6e-a72c-696619cf30ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:45.936843 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d653b-0650-4a6e-a72c-696619cf30ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:45.945423 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d653b-0650-4a6e-a72c-696619cf30ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:45.946840 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d653b-0650-4a6e-a72c-696619cf30ce service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:45.956149 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1382/2782] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:34:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:34:46.730798 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114d9f8-96ab-4c5d-b1e3-bf56f3c55330 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:46.741711 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114d9f8-96ab-4c5d-b1e3-bf56f3c55330 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:46.745760 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7ff77-043b-4eb7-a63b-bcb5810367d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:46.746370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7ff77-043b-4eb7-a63b-bcb5810367d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:46.746648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7ff77-043b-4eb7-a63b-bcb5810367d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:46.747047 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7ff77-043b-4eb7-a63b-bcb5810367d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:46.748945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0fd7ff77-043b-4eb7-a63b-bcb5810367d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:46.818543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7ff77-043b-4eb7-a63b-bcb5810367d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:46.820314 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7ff77-043b-4eb7-a63b-bcb5810367d1 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:46.833289 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1401/2783] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:34:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:34:47.616003 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0d625-bb73-429c-922b-6e00b3473514 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:47.616585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0d625-bb73-429c-922b-6e00b3473514 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:47.616585 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0d625-bb73-429c-922b-6e00b3473514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:47.636770 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-22f0d625-bb73-429c-922b-6e00b3473514 None None] MFA Rules not processed for user `19092e36ca6d4bc8a576df5e1c7c359d`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:34:47.637989 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-22f0d625-bb73-429c-922b-6e00b3473514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:47.645669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1383/2784] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:34:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:34:48.877500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7ff77-043b-4eb7-a63b-bcb5810367d1 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:34:48.879211 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0fd7ff77-043b-4eb7-a63b-bcb5810367d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:48.968419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7ff77-043b-4eb7-a63b-bcb5810367d1 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:34:48.974562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabbd13-4eed-47e7-8d9f-1fd29c268bfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:34:48.975176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabbd13-4eed-47e7-8d9f-1fd29c268bfa service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:34:48.975457 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabbd13-4eed-47e7-8d9f-1fd29c268bfa service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:34:48.975734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabbd13-4eed-47e7-8d9f-1fd29c268bfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:34:48.977492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9aabbd13-4eed-47e7-8d9f-1fd29c268bfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:34:48.991499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabbd13-4eed-47e7-8d9f-1fd29c268bfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:34:48.993193 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aabbd13-4eed-47e7-8d9f-1fd29c268bfa service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:34:49.019298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1402/2785] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:34:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:34:57.630996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=46 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:34:59.002242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=83 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:35:16.621966 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df244d25-cfb0-4d0e-95b2-12448e8540bf None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:35:16.622523 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df244d25-cfb0-4d0e-95b2-12448e8540bf None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:35:16.622523 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df244d25-cfb0-4d0e-95b2-12448e8540bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:35:16.642275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-df244d25-cfb0-4d0e-95b2-12448e8540bf None None] MFA Rules not processed for user `537431860e3d4c4fa629254f8c4c0cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:35:16.643470 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-df244d25-cfb0-4d0e-95b2-12448e8540bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:35:16.700602 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1384/2786] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:35:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:35:17.003072 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabbd13-4eed-47e7-8d9f-1fd29c268bfa service nova] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:35:17.005210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9aabbd13-4eed-47e7-8d9f-1fd29c268bfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:35:17.016429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aabbd13-4eed-47e7-8d9f-1fd29c268bfa service nova] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:35:17.019972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd232d26-12ea-4c37-8fef-3d7c5b9e87e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:35:17.020563 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd232d26-12ea-4c37-8fef-3d7c5b9e87e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:35:17.020837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd232d26-12ea-4c37-8fef-3d7c5b9e87e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:35:17.021109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd232d26-12ea-4c37-8fef-3d7c5b9e87e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:35:17.022792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-dd232d26-12ea-4c37-8fef-3d7c5b9e87e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:35:17.031953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd232d26-12ea-4c37-8fef-3d7c5b9e87e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:35:17.033367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd232d26-12ea-4c37-8fef-3d7c5b9e87e2 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:35:17.041726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1403/2787] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:35:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:35:26.694292 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=25 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:35:27.038180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=6 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:35:29.083492 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df244d25-cfb0-4d0e-95b2-12448e8540bf None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:35:29.084982 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-df244d25-cfb0-4d0e-95b2-12448e8540bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:35:29.100746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df244d25-cfb0-4d0e-95b2-12448e8540bf None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:35:29.106372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2172fc1-e93d-44b7-a202-d04058555f07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:35:29.106965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2172fc1-e93d-44b7-a202-d04058555f07 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:35:29.107247 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2172fc1-e93d-44b7-a202-d04058555f07 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:35:29.107519 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2172fc1-e93d-44b7-a202-d04058555f07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:35:29.109214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f2172fc1-e93d-44b7-a202-d04058555f07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:35:29.119309 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2172fc1-e93d-44b7-a202-d04058555f07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:35:29.120897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2172fc1-e93d-44b7-a202-d04058555f07 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:35:29.130460 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1385/2788] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:35:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:35:39.125656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=6 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:35:48.813248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd232d26-12ea-4c37-8fef-3d7c5b9e87e2 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:35:48.814634 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-dd232d26-12ea-4c37-8fef-3d7c5b9e87e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:35:48.882754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd232d26-12ea-4c37-8fef-3d7c5b9e87e2 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:35:48.887633 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3b3b0-0d40-4bb0-af14-bcd818d61533 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:35:48.888260 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3b3b0-0d40-4bb0-af14-bcd818d61533 service placement] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:35:48.888534 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3b3b0-0d40-4bb0-af14-bcd818d61533 service placement] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:35:48.888805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3b3b0-0d40-4bb0-af14-bcd818d61533 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:35:48.890495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a6d3b3b0-0d40-4bb0-af14-bcd818d61533 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:35:48.902209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3b3b0-0d40-4bb0-af14-bcd818d61533 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:35:48.903868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3b3b0-0d40-4bb0-af14-bcd818d61533 service placement] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:35:48.914196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1404/2789] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:35:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:35:58.909410 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=32 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:36:06.828310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2172fc1-e93d-44b7-a202-d04058555f07 service nova] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:36:06.838195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2172fc1-e93d-44b7-a202-d04058555f07 service nova] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:36:06.841751 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adde17-912d-4df3-a851-b8d9c97eec82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:36:06.842344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adde17-912d-4df3-a851-b8d9c97eec82 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:36:06.842623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adde17-912d-4df3-a851-b8d9c97eec82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:36:06.842897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4adde17-912d-4df3-a851-b8d9c97eec82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:36:06.844627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f4adde17-912d-4df3-a851-b8d9c97eec82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:36:06.856863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4adde17-912d-4df3-a851-b8d9c97eec82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:36:06.858302 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4adde17-912d-4df3-a851-b8d9c97eec82 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:36:06.867933 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1386/2790] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:36:13.865655 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3b3b0-0d40-4bb0-af14-bcd818d61533 service placement] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:36:13.867141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a6d3b3b0-0d40-4bb0-af14-bcd818d61533 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:36:13.881608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3b3b0-0d40-4bb0-af14-bcd818d61533 service placement] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:36:13.886127 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:36:13.886705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:36:13.886995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:36:13.887257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] PATH_INFO: `/v3/users/be10df0a4d1d42208c46615d6ddd970c` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:36:13.900605 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=be10df0a4d1d42208c46615d6ddd970c)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:36:13.902539 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:36:14.067498 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be10df0a4d1d42208c46615d6ddd970c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:14.122141 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be10df0a4d1d42208c46615d6ddd970c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:14.122804 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be10df0a4d1d42208c46615d6ddd970c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:14.123416 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be10df0a4d1d42208c46615d6ddd970c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:14.131024 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be10df0a4d1d42208c46615d6ddd970c'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:14.148529 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1405/2791] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:36:13 2022] DELETE /identity/v3/users/be10df0a4d1d42208c46615d6ddd970c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:36:14.385879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adde17-912d-4df3-a851-b8d9c97eec82 service neutron] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:36:14.396952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4adde17-912d-4df3-a851-b8d9c97eec82 service neutron] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:36:14.453670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:36:14.454308 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:36:14.454593 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:36:14.454874 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] PATH_INFO: `/v3/projects/1ccb39d9f1db4220ac101df9cb4572cf` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:36:14.466163 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ccb39d9f1db4220ac101df9cb4572cf)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:36:14.468129 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:36:14.578740 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ccb39d9f1db4220ac101df9cb4572cf', 'request_id': 'req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:14.641131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ccb39d9f1db4220ac101df9cb4572cf', 'request_id': 'req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:14.648685 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ccb39d9f1db4220ac101df9cb4572cf', 'request_id': 'req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:14.659197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] The token cache is being invalidate because project 1ccb39d9f1db4220ac101df9cb4572cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:36:14.659479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:14.660195 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:14.661716 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1387/2792] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:36:14 2022] DELETE /identity/v3/projects/1ccb39d9f1db4220ac101df9cb4572cf => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:36:14.665705 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:36:14.667116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:36:14.680163 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3306cb0-a246-4330-bd68-09599d9ba0ed admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:36:14.735621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e5a2a-eb3b-404d-a6f7-e28db2459110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:36:14.736268 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e5a2a-eb3b-404d-a6f7-e28db2459110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:36:14.736545 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e5a2a-eb3b-404d-a6f7-e28db2459110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:36:14.736822 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e5a2a-eb3b-404d-a6f7-e28db2459110 admin admin] PATH_INFO: `/v3/users/6904372579ae456c852f70b903c6603e` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:36:14.749552 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e5a2a-eb3b-404d-a6f7-e28db2459110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6904372579ae456c852f70b903c6603e)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:36:14.751460 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e5a2a-eb3b-404d-a6f7-e28db2459110 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:36:14.934443 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-2d3e5a2a-eb3b-404d-a6f7-e28db2459110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6904372579ae456c852f70b903c6603e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:14.937979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:36:15.007424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-2d3e5a2a-eb3b-404d-a6f7-e28db2459110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6904372579ae456c852f70b903c6603e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:15.008056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-2d3e5a2a-eb3b-404d-a6f7-e28db2459110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6904372579ae456c852f70b903c6603e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:15.008673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-2d3e5a2a-eb3b-404d-a6f7-e28db2459110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6904372579ae456c852f70b903c6603e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:15.016064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-2d3e5a2a-eb3b-404d-a6f7-e28db2459110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6904372579ae456c852f70b903c6603e'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:15.034590 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1406/2793] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:36:14 2022] DELETE /identity/v3/users/6904372579ae456c852f70b903c6603e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:36:15.272101 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:36:15.273660 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:36:15.289478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ec799-11e2-4e11-80c2-f41a9a8c4df2 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:36:15.350119 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:36:15.351154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:36:15.351154 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:36:15.351437 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] PATH_INFO: `/v3/projects/dc17de2d8943481dbccd43109bcb3e85` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:36:15.361979 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc17de2d8943481dbccd43109bcb3e85)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:36:15.363904 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:36:15.365798 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:36:15.489104 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc17de2d8943481dbccd43109bcb3e85', 'request_id': 'req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:15.595858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc17de2d8943481dbccd43109bcb3e85', 'request_id': 'req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:15.603724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc17de2d8943481dbccd43109bcb3e85', 'request_id': 'req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:15.614237 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] The token cache is being invalidate because project dc17de2d8943481dbccd43109bcb3e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:36:15.614548 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:15.615850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:36:15.617378 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1388/2794] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:36:15 2022] DELETE /identity/v3/projects/dc17de2d8943481dbccd43109bcb3e85 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:36:18.151427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e4ca6-232d-48eb-8f4b-d49257379178 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:36:18.151900 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e4ca6-232d-48eb-8f4b-d49257379178 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:36:18.152142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e4ca6-232d-48eb-8f4b-d49257379178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:36:18.172202 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-6c4e4ca6-232d-48eb-8f4b-d49257379178 None None] MFA Rules not processed for user `19092e36ca6d4bc8a576df5e1c7c359d`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:36:18.173337 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-6c4e4ca6-232d-48eb-8f4b-d49257379178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:36:18.228945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1407/2795] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:36:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:36:25.704849 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:36:28.221595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=37,keystone:DELETE=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:37:30.333397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:30.334866 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:30.391729 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6cf9c-cb6a-4682-b8eb-ca1c94db2427 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:30.395337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2477407-2c0d-45f8-acd7-7c796f5103bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:30.395968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2477407-2c0d-45f8-acd7-7c796f5103bd service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:30.396250 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2477407-2c0d-45f8-acd7-7c796f5103bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:30.396524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2477407-2c0d-45f8-acd7-7c796f5103bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:30.398223 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d2477407-2c0d-45f8-acd7-7c796f5103bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:30.468294 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2477407-2c0d-45f8-acd7-7c796f5103bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:30.469620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2477407-2c0d-45f8-acd7-7c796f5103bd service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:30.480811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1389/2796] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:37:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:37:30.645787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e4ca6-232d-48eb-8f4b-d49257379178 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:30.656004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e4ca6-232d-48eb-8f4b-d49257379178 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:30.659543 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5339fe-1646-44f1-b44c-c6948786653b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:30.660197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5339fe-1646-44f1-b44c-c6948786653b service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:30.660479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5339fe-1646-44f1-b44c-c6948786653b service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:30.660752 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5339fe-1646-44f1-b44c-c6948786653b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:30.662548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7b5339fe-1646-44f1-b44c-c6948786653b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:30.676181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5339fe-1646-44f1-b44c-c6948786653b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:30.677603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5339fe-1646-44f1-b44c-c6948786653b service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:30.689076 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1408/2797] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:37:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:37:32.414033 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2477407-2c0d-45f8-acd7-7c796f5103bd service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:32.416057 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d2477407-2c0d-45f8-acd7-7c796f5103bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:32.490482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2477407-2c0d-45f8-acd7-7c796f5103bd service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:32.495521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d22666-32b4-4742-b717-831985957b13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:32.496167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22666-32b4-4742-b717-831985957b13 service placement] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:32.496449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22666-32b4-4742-b717-831985957b13 service placement] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:32.496730 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d22666-32b4-4742-b717-831985957b13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:32.498977 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-79d22666-32b4-4742-b717-831985957b13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:32.511234 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d22666-32b4-4742-b717-831985957b13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:32.512905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d22666-32b4-4742-b717-831985957b13 service placement] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:32.523942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1390/2798] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:37:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:37:32.571603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5339fe-1646-44f1-b44c-c6948786653b service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:32.573603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7b5339fe-1646-44f1-b44c-c6948786653b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:32.641565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5339fe-1646-44f1-b44c-c6948786653b service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:32.646431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95b027-634d-498f-89a2-a322c602bb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:32.647057 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b027-634d-498f-89a2-a322c602bb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:32.647341 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b027-634d-498f-89a2-a322c602bb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:32.647618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95b027-634d-498f-89a2-a322c602bb63 admin admin] PATH_INFO: `/v3/users/19092e36ca6d4bc8a576df5e1c7c359d` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:32.648786 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b027-634d-498f-89a2-a322c602bb63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19092e36ca6d4bc8a576df5e1c7c359d)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:32.650778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95b027-634d-498f-89a2-a322c602bb63 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:32.826595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6e95b027-634d-498f-89a2-a322c602bb63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19092e36ca6d4bc8a576df5e1c7c359d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:32.897527 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6e95b027-634d-498f-89a2-a322c602bb63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19092e36ca6d4bc8a576df5e1c7c359d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:32.898138 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6e95b027-634d-498f-89a2-a322c602bb63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19092e36ca6d4bc8a576df5e1c7c359d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:32.898734 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6e95b027-634d-498f-89a2-a322c602bb63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19092e36ca6d4bc8a576df5e1c7c359d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:32.906431 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-6e95b027-634d-498f-89a2-a322c602bb63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19092e36ca6d4bc8a576df5e1c7c359d'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:32.923277 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1409/2799] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:37:32 2022] DELETE /identity/v3/users/19092e36ca6d4bc8a576df5e1c7c359d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:37:33.156809 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d22666-32b4-4742-b717-831985957b13 service placement] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:33.167753 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d22666-32b4-4742-b717-831985957b13 service placement] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:33.197861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:37:33.226224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:33.226855 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:33.227132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:33.227406 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] PATH_INFO: `/v3/projects/ba0a577cdbb44049934ace0e94520116` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:33.228573 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0a577cdbb44049934ace0e94520116)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:33.230473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:33.331938 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0a577cdbb44049934ace0e94520116', 'request_id': 'req-9939c5fd-dd49-4897-941d-4d27a3bb782c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:33.382852 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0a577cdbb44049934ace0e94520116', 'request_id': 'req-9939c5fd-dd49-4897-941d-4d27a3bb782c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:33.389594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0a577cdbb44049934ace0e94520116', 'request_id': 'req-9939c5fd-dd49-4897-941d-4d27a3bb782c'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:33.400054 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] The token cache is being invalidate because project ba0a577cdbb44049934ace0e94520116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:37:33.400224 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:33.400941 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-9939c5fd-dd49-4897-941d-4d27a3bb782c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:33.402440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1391/2800] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:37:33 2022] DELETE /identity/v3/projects/ba0a577cdbb44049934ace0e94520116 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:37:33.407470 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557fce-b363-4a9c-b1c9-948537238097 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:33.407757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557fce-b363-4a9c-b1c9-948537238097 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:33.408070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca557fce-b363-4a9c-b1c9-948537238097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:33.427248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-ca557fce-b363-4a9c-b1c9-948537238097 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:37:33.428510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-ca557fce-b363-4a9c-b1c9-948537238097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:33.492513 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1410/2801] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:37:33.501647 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b1f06-589b-40a9-8344-1eff90b16599 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:33.502127 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b1f06-589b-40a9-8344-1eff90b16599 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:33.502279 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b1f06-589b-40a9-8344-1eff90b16599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:33.522560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-9f9b1f06-589b-40a9-8344-1eff90b16599 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:37:33.523794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9f9b1f06-589b-40a9-8344-1eff90b16599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:33.531649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1392/2802] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:37:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:37:33.535883 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557fce-b363-4a9c-b1c9-948537238097 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:33.545864 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca557fce-b363-4a9c-b1c9-948537238097 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:33.550490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002a557-13b4-489a-a8b0-da1b0b4e56f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:33.551079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002a557-13b4-489a-a8b0-da1b0b4e56f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:33.551358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002a557-13b4-489a-a8b0-da1b0b4e56f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:33.551632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a002a557-13b4-489a-a8b0-da1b0b4e56f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:33.552303 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002a557-13b4-489a-a8b0-da1b0b4e56f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:37:33.552607 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002a557-13b4-489a-a8b0-da1b0b4e56f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:33.554188 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a002a557-13b4-489a-a8b0-da1b0b4e56f5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:33.555623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:37:33.566526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1411/2803] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:37:33.572222 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b1f06-589b-40a9-8344-1eff90b16599 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:33.581333 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b1f06-589b-40a9-8344-1eff90b16599 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:33.586218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48ba57-3a91-4498-847a-6d745fda0f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:33.587110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48ba57-3a91-4498-847a-6d745fda0f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:33.587419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48ba57-3a91-4498-847a-6d745fda0f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:33.587705 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48ba57-3a91-4498-847a-6d745fda0f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:33.588368 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48ba57-3a91-4498-847a-6d745fda0f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:33.590269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48ba57-3a91-4498-847a-6d745fda0f37 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:33.704935 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1393/2804] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:37:33 2022] POST /identity/v3/projects => generated 391 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:37:33.709665 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002a557-13b4-489a-a8b0-da1b0b4e56f5 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:33.720881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a002a557-13b4-489a-a8b0-da1b0b4e56f5 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:33.782119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4977e-510f-46a8-90aa-c777167ba3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:33.782824 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4977e-510f-46a8-90aa-c777167ba3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:33.783100 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4977e-510f-46a8-90aa-c777167ba3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:33.783371 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4977e-510f-46a8-90aa-c777167ba3e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:33.784043 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4977e-510f-46a8-90aa-c777167ba3e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:33.786135 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4977e-510f-46a8-90aa-c777167ba3e3 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:33.891616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1412/2805] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:37:33 2022] POST /identity/v3/users => generated 296 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:37:33.895706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48ba57-3a91-4498-847a-6d745fda0f37 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:33.906146 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48ba57-3a91-4498-847a-6d745fda0f37 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:33.910792 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf789c-4547-430f-bc6d-2fa3b3082d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:33.911385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf789c-4547-430f-bc6d-2fa3b3082d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:33.911668 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf789c-4547-430f-bc6d-2fa3b3082d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:33.911981 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bf789c-4547-430f-bc6d-2fa3b3082d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:33.912626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf789c-4547-430f-bc6d-2fa3b3082d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:33.914226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bf789c-4547-430f-bc6d-2fa3b3082d44 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:33.925031 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1394/2806] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:37:33.928881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4977e-510f-46a8-90aa-c777167ba3e3 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:33.939572 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4977e-510f-46a8-90aa-c777167ba3e3 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:33.944603 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc0fc5-702d-4c6a-bd92-c4a9829d6aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:33.947318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc0fc5-702d-4c6a-bd92-c4a9829d6aff admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:33.947318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc0fc5-702d-4c6a-bd92-c4a9829d6aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:33.947318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc0fc5-702d-4c6a-bd92-c4a9829d6aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:33.947318 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc0fc5-702d-4c6a-bd92-c4a9829d6aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:33.947939 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc0fc5-702d-4c6a-bd92-c4a9829d6aff admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:33.959502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1413/2807] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:37:33.963711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf789c-4547-430f-bc6d-2fa3b3082d44 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:33.974254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bf789c-4547-430f-bc6d-2fa3b3082d44 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:33.978864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d9f50-a314-44de-99f2-fb990f850617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:33.979508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d9f50-a314-44de-99f2-fb990f850617 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:33.979822 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d9f50-a314-44de-99f2-fb990f850617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:33.980105 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d9f50-a314-44de-99f2-fb990f850617 admin admin] PATH_INFO: `/v3/projects/490f3056dbad4e478e2ac7f687893505/users/2e7010e583744f0a9f48978a81413a07/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:33.995545 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d9f50-a314-44de-99f2-fb990f850617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=490f3056dbad4e478e2ac7f687893505, user_id=2e7010e583744f0a9f48978a81413a07, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:33.998244 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d9f50-a314-44de-99f2-fb990f850617 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:34.057664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1395/2808] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:37:33 2022] PUT /identity/v3/projects/490f3056dbad4e478e2ac7f687893505/users/2e7010e583744f0a9f48978a81413a07/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:37:34.068316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd6e6a-79b2-42d5-95c3-58361da2b97a None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:34.068601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd6e6a-79b2-42d5-95c3-58361da2b97a None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:34.068894 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd6e6a-79b2-42d5-95c3-58361da2b97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:34.098766 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-79bd6e6a-79b2-42d5-95c3-58361da2b97a None None] MFA Rules not processed for user `2e7010e583744f0a9f48978a81413a07`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:37:34.100101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-79bd6e6a-79b2-42d5-95c3-58361da2b97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:34.162757 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1414/2809] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:37:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:37:34.174319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d9f50-a314-44de-99f2-fb990f850617 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:34.176145 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-dd3d9f50-a314-44de-99f2-fb990f850617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:34.231651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d9f50-a314-44de-99f2-fb990f850617 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:34.236082 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8843b-f8bb-430e-bf28-80a472d6d1e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:34.236680 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8843b-f8bb-430e-bf28-80a472d6d1e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:34.236957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8843b-f8bb-430e-bf28-80a472d6d1e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:34.237300 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8843b-f8bb-430e-bf28-80a472d6d1e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:34.245962 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8843b-f8bb-430e-bf28-80a472d6d1e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:34.247400 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8843b-f8bb-430e-bf28-80a472d6d1e3 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:34.251990 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:37:34.256510 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1396/2810] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:37:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:37:42.755885 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf1c66-1051-471e-a9f2-6d83703438aa None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:42.756380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf1c66-1051-471e-a9f2-6d83703438aa None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:42.756499 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf1c66-1051-471e-a9f2-6d83703438aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:42.775355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-aabf1c66-1051-471e-a9f2-6d83703438aa None None] MFA Rules not processed for user `537431860e3d4c4fa629254f8c4c0cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:37:42.776577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-aabf1c66-1051-471e-a9f2-6d83703438aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:42.830495 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1415/2811] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:37:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:37:52.823449 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:37:58.536219 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8843b-f8bb-430e-bf28-80a472d6d1e3 service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:58.634966 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8843b-f8bb-430e-bf28-80a472d6d1e3 service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:58.639900 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:58.640539 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:58.640820 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:58.641089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] PATH_INFO: `/v3/users/2e7010e583744f0a9f48978a81413a07` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:58.642479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e7010e583744f0a9f48978a81413a07)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:58.644554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:58.855572 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e7010e583744f0a9f48978a81413a07'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:58.938189 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e7010e583744f0a9f48978a81413a07'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:58.939072 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e7010e583744f0a9f48978a81413a07'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:58.939902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e7010e583744f0a9f48978a81413a07'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:58.966659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e7010e583744f0a9f48978a81413a07'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:58.997529 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1397/2812] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:37:58 2022] DELETE /identity/v3/users/2e7010e583744f0a9f48978a81413a07 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:37:59.002901 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a190e9-dbee-4e60-aa17-6e93101b6dbe None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:59.004787 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a190e9-dbee-4e60-aa17-6e93101b6dbe None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:59.005279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a190e9-dbee-4e60-aa17-6e93101b6dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:59.029878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b2a190e9-dbee-4e60-aa17-6e93101b6dbe None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:37:59.031754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b2a190e9-dbee-4e60-aa17-6e93101b6dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:59.093614 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1416/2813] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:37:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:37:59.103301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:59.104731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:59.166407 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0247c-0097-4c99-9840-9ad91487f71b admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:59.167474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117a91b-4cf9-48cb-ac21-d23bf63273f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:59.168125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117a91b-4cf9-48cb-ac21-d23bf63273f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:59.168401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117a91b-4cf9-48cb-ac21-d23bf63273f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:59.168669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117a91b-4cf9-48cb-ac21-d23bf63273f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:59.177482 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117a91b-4cf9-48cb-ac21-d23bf63273f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:59.178886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117a91b-4cf9-48cb-ac21-d23bf63273f9 service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:59.188215 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1398/2814] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:37:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:37:59.818658 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a190e9-dbee-4e60-aa17-6e93101b6dbe None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:37:59.820390 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b2a190e9-dbee-4e60-aa17-6e93101b6dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:37:59.836528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a190e9-dbee-4e60-aa17-6e93101b6dbe None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:37:59.844284 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:37:59.845103 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:37:59.845433 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:37:59.845716 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] PATH_INFO: `/v3/projects/490f3056dbad4e478e2ac7f687893505` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:37:59.846921 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] RBAC: Authorizing `identity:delete_project(project_id=490f3056dbad4e478e2ac7f687893505)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:37:59.848871 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:37:59.957108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '490f3056dbad4e478e2ac7f687893505', 'request_id': 'req-11573169-a1d1-4cde-b6c9-d1768253131f'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:37:59.960618 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '490f3056dbad4e478e2ac7f687893505', 'request_id': 'req-11573169-a1d1-4cde-b6c9-d1768253131f'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:00.033767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '490f3056dbad4e478e2ac7f687893505', 'request_id': 'req-11573169-a1d1-4cde-b6c9-d1768253131f'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:00.045313 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] The token cache is being invalidate because project 490f3056dbad4e478e2ac7f687893505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:38:00.055128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:00.055128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:00.055128 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1417/2815] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:37:59 2022] DELETE /identity/v3/projects/490f3056dbad4e478e2ac7f687893505 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:38:01.253151 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978c49d-a1f9-4040-a794-b23e0061c114 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:38:01.253617 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978c49d-a1f9-4040-a794-b23e0061c114 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:38:01.253739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978c49d-a1f9-4040-a794-b23e0061c114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:38:01.274580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-b978c49d-a1f9-4040-a794-b23e0061c114 None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:38:01.275845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b978c49d-a1f9-4040-a794-b23e0061c114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:38:01.295150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:38:01.334266 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1399/2816] 10.222.0.15 () {58 vars in 967 bytes} [Wed Sep 7 16:38:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:38:01.375635 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:38:01.377619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:38:01.449395 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11573169-a1d1-4cde-b6c9-d1768253131f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:38:01.455096 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5716e9-be63-4d5a-90ce-4392dc30170f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:38:01.455740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5716e9-be63-4d5a-90ce-4392dc30170f service swift] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:38:01.456051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5716e9-be63-4d5a-90ce-4392dc30170f service swift] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:38:01.456328 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5716e9-be63-4d5a-90ce-4392dc30170f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:38:01.464650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5716e9-be63-4d5a-90ce-4392dc30170f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:38:01.466177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5716e9-be63-4d5a-90ce-4392dc30170f service swift] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:38:01.475052 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1418/2817] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:38:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:38:09.517897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978c49d-a1f9-4040-a794-b23e0061c114 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:38:09.519357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b978c49d-a1f9-4040-a794-b23e0061c114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:38:09.595599 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978c49d-a1f9-4040-a794-b23e0061c114 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:38:09.600725 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281cd2-8e19-4d43-bb86-a53050f2cba8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:38:09.601340 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281cd2-8e19-4d43-bb86-a53050f2cba8 service placement] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:38:09.601623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281cd2-8e19-4d43-bb86-a53050f2cba8 service placement] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:38:09.601901 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281cd2-8e19-4d43-bb86-a53050f2cba8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:38:09.603634 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-f6281cd2-8e19-4d43-bb86-a53050f2cba8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:38:09.617364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6281cd2-8e19-4d43-bb86-a53050f2cba8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:38:09.619422 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6281cd2-8e19-4d43-bb86-a53050f2cba8 service placement] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:38:09.625770 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5716e9-be63-4d5a-90ce-4392dc30170f service swift] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:38:09.628948 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-cd5716e9-be63-4d5a-90ce-4392dc30170f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:38:09.631588 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1400/2818] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:38:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:38:09.648641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:38:09.700031 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5716e9-be63-4d5a-90ce-4392dc30170f service swift] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:38:09.704500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:38:09.705111 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:38:09.705389 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:38:09.705662 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] PATH_INFO: `/v3/users/b7fdcf3f310b455d925e277ab48828ec` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:38:09.706784 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7fdcf3f310b455d925e277ab48828ec)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:38:09.708719 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:38:09.829720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7fdcf3f310b455d925e277ab48828ec'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:09.874248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7fdcf3f310b455d925e277ab48828ec'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:09.874859 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7fdcf3f310b455d925e277ab48828ec'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:09.875483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7fdcf3f310b455d925e277ab48828ec'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:09.882345 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7fdcf3f310b455d925e277ab48828ec'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:09.899476 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1419/2819] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:38:09 2022] DELETE /identity/v3/users/b7fdcf3f310b455d925e277ab48828ec => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:38:10.143456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281cd2-8e19-4d43-bb86-a53050f2cba8 service placement] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:38:10.154102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281cd2-8e19-4d43-bb86-a53050f2cba8 service placement] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:38:10.219375 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:38:10.220028 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:38:10.220312 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:38:10.220589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] PATH_INFO: `/v3/projects/3ec6a0ab035148cd9466f28ad3f1ae59` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:38:10.221747 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec6a0ab035148cd9466f28ad3f1ae59)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:38:10.223637 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:38:10.336028 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec6a0ab035148cd9466f28ad3f1ae59', 'request_id': 'req-afb35a61-a864-477d-8cb5-6565cd358fd7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:10.410468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec6a0ab035148cd9466f28ad3f1ae59', 'request_id': 'req-afb35a61-a864-477d-8cb5-6565cd358fd7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:10.412566 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:38:10.417155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec6a0ab035148cd9466f28ad3f1ae59', 'request_id': 'req-afb35a61-a864-477d-8cb5-6565cd358fd7'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:10.427268 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] The token cache is being invalidate because project 3ec6a0ab035148cd9466f28ad3f1ae59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:38:10.427558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:10.428210 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:10.429679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1401/2820] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:38:10 2022] DELETE /identity/v3/projects/3ec6a0ab035148cd9466f28ad3f1ae59 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:38:10.433653 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:38:10.435144 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:38:10.448160 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f405000-a0f5-41e8-a093-5daf95621133 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:38:10.502783 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:38:10.503402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:38:10.503681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:38:10.503979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] PATH_INFO: `/v3/users/0769a62c8f644d93881d4fa794931fa8` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:38:10.505078 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0769a62c8f644d93881d4fa794931fa8)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:38:10.506979 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:38:10.673699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:38:10.682268 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0769a62c8f644d93881d4fa794931fa8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:10.781404 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0769a62c8f644d93881d4fa794931fa8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:10.782064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0769a62c8f644d93881d4fa794931fa8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:10.782671 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0769a62c8f644d93881d4fa794931fa8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:10.789650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0769a62c8f644d93881d4fa794931fa8'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:10.806881 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1420/2821] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:38:10 2022] DELETE /identity/v3/users/0769a62c8f644d93881d4fa794931fa8 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:38:11.047517 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:38:11.049232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:38:11.062648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb35a61-a864-477d-8cb5-6565cd358fd7 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:38:11.118924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:38:11.119556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:38:11.119875 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:38:11.120155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] PATH_INFO: `/v3/projects/6a79d8aa0fd847cd8d7fc19357784ed0` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:38:11.121273 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a79d8aa0fd847cd8d7fc19357784ed0)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:38:11.123166 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:38:11.227604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a79d8aa0fd847cd8d7fc19357784ed0', 'request_id': 'req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:11.302238 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a79d8aa0fd847cd8d7fc19357784ed0', 'request_id': 'req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:11.308365 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a79d8aa0fd847cd8d7fc19357784ed0', 'request_id': 'req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:11.318537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] The token cache is being invalidate because project 6a79d8aa0fd847cd8d7fc19357784ed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:38:11.322996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:11.322996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:38:11.322996 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1402/2822] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:38:11 2022] DELETE /identity/v3/projects/6a79d8aa0fd847cd8d7fc19357784ed0 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:38:20.868296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:38:21.315613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=55,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:38:42.339274 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:38:42.340723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:38:42.395754 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad6d291-58d2-41b3-b1ae-df0828c45860 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:38:42.441511 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47af8dc-9020-4e6f-ac96-027815f6bf77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:38:42.442108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af8dc-9020-4e6f-ac96-027815f6bf77 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:38:42.442382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af8dc-9020-4e6f-ac96-027815f6bf77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:38:42.442652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47af8dc-9020-4e6f-ac96-027815f6bf77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:38:42.444348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b47af8dc-9020-4e6f-ac96-027815f6bf77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:38:42.453423 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47af8dc-9020-4e6f-ac96-027815f6bf77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:38:42.454854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47af8dc-9020-4e6f-ac96-027815f6bf77 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:38:42.463008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1421/2823] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:38:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:38:52.459068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=46 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:39:32.950559 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:32.952113 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:33.061864 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c45f9-42cb-4fc2-b6be-d4576caaa71a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:33.068457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd29bd7-d710-40d2-be1d-47207a7a8e04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:33.071547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd29bd7-d710-40d2-be1d-47207a7a8e04 service nova] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:33.071547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd29bd7-d710-40d2-be1d-47207a7a8e04 service nova] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:33.071547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd29bd7-d710-40d2-be1d-47207a7a8e04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:33.072605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9fd29bd7-d710-40d2-be1d-47207a7a8e04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:33.092487 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd29bd7-d710-40d2-be1d-47207a7a8e04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:33.094992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd29bd7-d710-40d2-be1d-47207a7a8e04 service nova] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:33.115997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1403/2824] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:39:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:39:39.941185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47af8dc-9020-4e6f-ac96-027815f6bf77 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:39.942626 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b47af8dc-9020-4e6f-ac96-027815f6bf77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:39.953806 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47af8dc-9020-4e6f-ac96-027815f6bf77 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:39.957425 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4e928-4f8a-4ce8-9742-bb207ce6c00c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:39.958019 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4e928-4f8a-4ce8-9742-bb207ce6c00c service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:39.958299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4e928-4f8a-4ce8-9742-bb207ce6c00c service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:39.958575 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4e928-4f8a-4ce8-9742-bb207ce6c00c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:39.960831 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-5db4e928-4f8a-4ce8-9742-bb207ce6c00c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:40.023315 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4e928-4f8a-4ce8-9742-bb207ce6c00c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:40.024795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4e928-4f8a-4ce8-9742-bb207ce6c00c service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:40.034310 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1422/2825] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:39:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:39:43.106321 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=34 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:39:43.388290 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd29bd7-d710-40d2-be1d-47207a7a8e04 service nova] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:43.389718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-9fd29bd7-d710-40d2-be1d-47207a7a8e04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:43.405069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd29bd7-d710-40d2-be1d-47207a7a8e04 service nova] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:43.410232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a6505-6b2e-406c-a784-d9f91e1cd6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:43.411000 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a6505-6b2e-406c-a784-d9f91e1cd6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:43.411506 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a6505-6b2e-406c-a784-d9f91e1cd6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:43.411968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a6505-6b2e-406c-a784-d9f91e1cd6ba admin admin] PATH_INFO: `/v3/users/537431860e3d4c4fa629254f8c4c0cf4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:43.413153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a6505-6b2e-406c-a784-d9f91e1cd6ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=537431860e3d4c4fa629254f8c4c0cf4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:43.416133 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940a6505-6b2e-406c-a784-d9f91e1cd6ba admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:43.548547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-940a6505-6b2e-406c-a784-d9f91e1cd6ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '537431860e3d4c4fa629254f8c4c0cf4'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:43.592561 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-940a6505-6b2e-406c-a784-d9f91e1cd6ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '537431860e3d4c4fa629254f8c4c0cf4'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:43.593187 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-940a6505-6b2e-406c-a784-d9f91e1cd6ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '537431860e3d4c4fa629254f8c4c0cf4'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:43.593784 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-940a6505-6b2e-406c-a784-d9f91e1cd6ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '537431860e3d4c4fa629254f8c4c0cf4'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:43.600711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-940a6505-6b2e-406c-a784-d9f91e1cd6ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '537431860e3d4c4fa629254f8c4c0cf4'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:43.617827 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1404/2826] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:39:43 2022] DELETE /identity/v3/users/537431860e3d4c4fa629254f8c4c0cf4 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:39:43.868631 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4e928-4f8a-4ce8-9742-bb207ce6c00c service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:43.882648 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4e928-4f8a-4ce8-9742-bb207ce6c00c service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:43.937619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:43.938255 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:43.938537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:43.938819 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] PATH_INFO: `/v3/projects/b7b0bd00008a4ceaa2979a6ec6a10dbd` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:43.939970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7b0bd00008a4ceaa2979a6ec6a10dbd)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:43.941872 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:44.056830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7b0bd00008a4ceaa2979a6ec6a10dbd', 'request_id': 'req-8ed61393-39ef-4e25-a798-33ce4aab7deb'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:44.062742 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7b0bd00008a4ceaa2979a6ec6a10dbd', 'request_id': 'req-8ed61393-39ef-4e25-a798-33ce4aab7deb'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:44.129461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7b0bd00008a4ceaa2979a6ec6a10dbd', 'request_id': 'req-8ed61393-39ef-4e25-a798-33ce4aab7deb'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:44.141068 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] The token cache is being invalidate because project b7b0bd00008a4ceaa2979a6ec6a10dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:39:44.141355 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:44.141968 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8ed61393-39ef-4e25-a798-33ce4aab7deb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:44.143419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1423/2827] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:39:43 2022] DELETE /identity/v3/projects/b7b0bd00008a4ceaa2979a6ec6a10dbd => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:39:44.148494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dcbb6-f407-43eb-b767-e321da50c240 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:44.148980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dcbb6-f407-43eb-b767-e321da50c240 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:44.149099 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581dcbb6-f407-43eb-b767-e321da50c240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:44.168657 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-581dcbb6-f407-43eb-b767-e321da50c240 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:39:44.169836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-581dcbb6-f407-43eb-b767-e321da50c240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:44.228328 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1405/2828] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:39:44.235726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9c006-0f83-4ec1-b270-fbaf7fb9c202 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:44.236036 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9c006-0f83-4ec1-b270-fbaf7fb9c202 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:44.236312 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9c006-0f83-4ec1-b270-fbaf7fb9c202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:44.255930 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-a0c9c006-0f83-4ec1-b270-fbaf7fb9c202 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:39:44.257119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-a0c9c006-0f83-4ec1-b270-fbaf7fb9c202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:44.264854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1424/2829] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:39:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:39:44.268759 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dcbb6-f407-43eb-b767-e321da50c240 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:44.278280 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581dcbb6-f407-43eb-b767-e321da50c240 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:44.282807 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725d802-15c9-426a-9c4a-34fa9c922023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:44.283389 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725d802-15c9-426a-9c4a-34fa9c922023 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:44.283665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725d802-15c9-426a-9c4a-34fa9c922023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:44.283970 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725d802-15c9-426a-9c4a-34fa9c922023 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:44.284604 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725d802-15c9-426a-9c4a-34fa9c922023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:39:44.284905 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725d802-15c9-426a-9c4a-34fa9c922023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:44.286503 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725d802-15c9-426a-9c4a-34fa9c922023 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:44.298902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1406/2830] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:39:44.303558 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9c006-0f83-4ec1-b270-fbaf7fb9c202 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:44.312502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9c006-0f83-4ec1-b270-fbaf7fb9c202 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:44.317051 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521442f2-0523-451c-a054-aee31cff7a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:44.317627 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521442f2-0523-451c-a054-aee31cff7a8e admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:44.317899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521442f2-0523-451c-a054-aee31cff7a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:44.318165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521442f2-0523-451c-a054-aee31cff7a8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:44.318820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521442f2-0523-451c-a054-aee31cff7a8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:44.320998 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521442f2-0523-451c-a054-aee31cff7a8e admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:44.335899 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:39:44.439613 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1425/2831] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:39:44 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:39:44.443688 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725d802-15c9-426a-9c4a-34fa9c922023 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:44.453947 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725d802-15c9-426a-9c4a-34fa9c922023 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:44.510531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37931db4-9e39-47b4-8949-1db03da60146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:44.511130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37931db4-9e39-47b4-8949-1db03da60146 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:44.511408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37931db4-9e39-47b4-8949-1db03da60146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:44.511679 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37931db4-9e39-47b4-8949-1db03da60146 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:44.512337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37931db4-9e39-47b4-8949-1db03da60146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:44.514232 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37931db4-9e39-47b4-8949-1db03da60146 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:44.522050 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:39:44.633028 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1407/2832] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:39:44 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:39:44.637125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521442f2-0523-451c-a054-aee31cff7a8e admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:44.648982 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521442f2-0523-451c-a054-aee31cff7a8e admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:44.653578 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06be1-c6dd-41e0-88f9-3ccc1786b1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:44.654177 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06be1-c6dd-41e0-88f9-3ccc1786b1eb admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:44.654537 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06be1-c6dd-41e0-88f9-3ccc1786b1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:44.654837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd06be1-c6dd-41e0-88f9-3ccc1786b1eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:44.655436 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd06be1-c6dd-41e0-88f9-3ccc1786b1eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:44.657892 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd06be1-c6dd-41e0-88f9-3ccc1786b1eb admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:44.669213 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1426/2833] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:39:44.674364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37931db4-9e39-47b4-8949-1db03da60146 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:44.689086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37931db4-9e39-47b4-8949-1db03da60146 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:44.722178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b59dda-0b46-463e-8a4e-2c04578719fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:44.722178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b59dda-0b46-463e-8a4e-2c04578719fe admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:44.722178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b59dda-0b46-463e-8a4e-2c04578719fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:44.722178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b59dda-0b46-463e-8a4e-2c04578719fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:44.722945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b59dda-0b46-463e-8a4e-2c04578719fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:44.722945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b59dda-0b46-463e-8a4e-2c04578719fe admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:44.722945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1408/2834] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:39:44.725411 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06be1-c6dd-41e0-88f9-3ccc1786b1eb admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:44.736412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd06be1-c6dd-41e0-88f9-3ccc1786b1eb admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:44.741385 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddcc9c-fe7a-474d-8bc7-2f424cabc76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:44.742586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddcc9c-fe7a-474d-8bc7-2f424cabc76a admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:44.742866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddcc9c-fe7a-474d-8bc7-2f424cabc76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:44.743142 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ddcc9c-fe7a-474d-8bc7-2f424cabc76a admin admin] PATH_INFO: `/v3/projects/aa6573ed382047eb9274f889a7d52dc5/users/cf0a60fdd9e04af5a7f7ae7c7370b26d/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:44.758020 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddcc9c-fe7a-474d-8bc7-2f424cabc76a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa6573ed382047eb9274f889a7d52dc5, user_id=cf0a60fdd9e04af5a7f7ae7c7370b26d, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:44.760713 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ddcc9c-fe7a-474d-8bc7-2f424cabc76a admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:44.849415 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1427/2835] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:39:44 2022] PUT /identity/v3/projects/aa6573ed382047eb9274f889a7d52dc5/users/cf0a60fdd9e04af5a7f7ae7c7370b26d/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:39:44.860088 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564db1-09bc-47d3-8651-99e60b9bbfbe None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:44.860370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564db1-09bc-47d3-8651-99e60b9bbfbe None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:44.860643 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564db1-09bc-47d3-8651-99e60b9bbfbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:44.888426 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-e7564db1-09bc-47d3-8651-99e60b9bbfbe None None] MFA Rules not processed for user `cf0a60fdd9e04af5a7f7ae7c7370b26d`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:39:44.889640 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e7564db1-09bc-47d3-8651-99e60b9bbfbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:44.952271 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1409/2836] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:39:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:39:44.963185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddcc9c-fe7a-474d-8bc7-2f424cabc76a admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:44.965042 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-51ddcc9c-fe7a-474d-8bc7-2f424cabc76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:45.036593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ddcc9c-fe7a-474d-8bc7-2f424cabc76a admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:45.042594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00e727-0210-4eed-bccf-50e146c60b46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:45.043197 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00e727-0210-4eed-bccf-50e146c60b46 service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:45.043501 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00e727-0210-4eed-bccf-50e146c60b46 service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:45.043793 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00e727-0210-4eed-bccf-50e146c60b46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:45.051253 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00e727-0210-4eed-bccf-50e146c60b46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:45.052857 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a00e727-0210-4eed-bccf-50e146c60b46 service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:45.062017 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1428/2837] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:39:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:39:54.197955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7564db1-09bc-47d3-8651-99e60b9bbfbe None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:54.259541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7564db1-09bc-47d3-8651-99e60b9bbfbe None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:54.264420 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:54.264994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:54.265276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:54.265554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] PATH_INFO: `/v3/users/cf0a60fdd9e04af5a7f7ae7c7370b26d` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:54.266722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0a60fdd9e04af5a7f7ae7c7370b26d)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:54.268711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:54.477284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0a60fdd9e04af5a7f7ae7c7370b26d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:54.552580 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0a60fdd9e04af5a7f7ae7c7370b26d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:54.553131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0a60fdd9e04af5a7f7ae7c7370b26d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:54.553739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0a60fdd9e04af5a7f7ae7c7370b26d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:54.561435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0a60fdd9e04af5a7f7ae7c7370b26d'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:54.567058 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:39:54.580150 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1410/2838] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:39:54 2022] DELETE /identity/v3/users/cf0a60fdd9e04af5a7f7ae7c7370b26d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:39:54.585720 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb836-8f18-4b73-840c-0fd70f87a505 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:54.586223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb836-8f18-4b73-840c-0fd70f87a505 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:54.586683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb836-8f18-4b73-840c-0fd70f87a505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:54.607947 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-0f7bb836-8f18-4b73-840c-0fd70f87a505 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:39:54.609203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0f7bb836-8f18-4b73-840c-0fd70f87a505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:54.670021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1429/2839] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:39:54.679364 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:54.680799 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:54.741337 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da5fcc-d09b-4c25-857c-3ad7e5ba71aa admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:54.745789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a243e8-d371-44dc-8bd8-ceed52806fcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:54.745789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a243e8-d371-44dc-8bd8-ceed52806fcf service neutron] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:54.745789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a243e8-d371-44dc-8bd8-ceed52806fcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:54.746141 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a243e8-d371-44dc-8bd8-ceed52806fcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:54.755077 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a243e8-d371-44dc-8bd8-ceed52806fcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:54.756811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a243e8-d371-44dc-8bd8-ceed52806fcf service neutron] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:54.767534 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1411/2840] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:39:55.193790 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb836-8f18-4b73-840c-0fd70f87a505 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:55.197738 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0f7bb836-8f18-4b73-840c-0fd70f87a505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:55.208593 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb836-8f18-4b73-840c-0fd70f87a505 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:55.212989 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:55.213699 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:55.213970 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:55.214245 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] PATH_INFO: `/v3/projects/aa6573ed382047eb9274f889a7d52dc5` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:55.215366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa6573ed382047eb9274f889a7d52dc5)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:55.217304 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:55.307040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:39:55.312917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa6573ed382047eb9274f889a7d52dc5', 'request_id': 'req-e02fa06c-2251-4845-8d39-59e848f2b510'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:55.318228 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa6573ed382047eb9274f889a7d52dc5', 'request_id': 'req-e02fa06c-2251-4845-8d39-59e848f2b510'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:55.392093 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa6573ed382047eb9274f889a7d52dc5', 'request_id': 'req-e02fa06c-2251-4845-8d39-59e848f2b510'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:55.403427 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] The token cache is being invalidate because project aa6573ed382047eb9274f889a7d52dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81775) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:39:55.403704 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:55.404358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-e02fa06c-2251-4845-8d39-59e848f2b510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:39:55.405781 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1430/2841] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:39:55 2022] DELETE /identity/v3/projects/aa6573ed382047eb9274f889a7d52dc5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:39:55.410701 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c5e75-2773-471f-bd93-3f661dadb28b None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:55.411205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c5e75-2773-471f-bd93-3f661dadb28b None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:55.411319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c5e75-2773-471f-bd93-3f661dadb28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:55.431703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-2d5c5e75-2773-471f-bd93-3f661dadb28b None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:39:55.432923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-2d5c5e75-2773-471f-bd93-3f661dadb28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:55.493362 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1412/2842] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:39:55.550841 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab79d8e-1de3-47d2-b245-4216893cffa6 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:55.551125 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab79d8e-1de3-47d2-b245-4216893cffa6 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:55.551402 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab79d8e-1de3-47d2-b245-4216893cffa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:55.572176 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-bab79d8e-1de3-47d2-b245-4216893cffa6 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:39:55.573373 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-bab79d8e-1de3-47d2-b245-4216893cffa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:55.581079 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1431/2843] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:39:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:39:55.585194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c5e75-2773-471f-bd93-3f661dadb28b None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:55.594789 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c5e75-2773-471f-bd93-3f661dadb28b None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:55.599268 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be4826-63ab-49b1-a759-9c400ca7c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:55.599886 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be4826-63ab-49b1-a759-9c400ca7c1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:55.600171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be4826-63ab-49b1-a759-9c400ca7c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:55.600445 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39be4826-63ab-49b1-a759-9c400ca7c1a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:55.601595 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be4826-63ab-49b1-a759-9c400ca7c1a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81776) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:39:55.601899 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be4826-63ab-49b1-a759-9c400ca7c1a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:55.603480 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39be4826-63ab-49b1-a759-9c400ca7c1a5 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:55.614965 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1413/2844] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:39:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:39:55.619424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab79d8e-1de3-47d2-b245-4216893cffa6 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:55.628530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab79d8e-1de3-47d2-b245-4216893cffa6 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:55.633148 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bce199-a2ac-4963-b1b6-6703e5e9018b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:55.633741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce199-a2ac-4963-b1b6-6703e5e9018b admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:55.634022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce199-a2ac-4963-b1b6-6703e5e9018b admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:55.634297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce199-a2ac-4963-b1b6-6703e5e9018b admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:55.634952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bce199-a2ac-4963-b1b6-6703e5e9018b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:55.636878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bce199-a2ac-4963-b1b6-6703e5e9018b admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:55.713413 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1432/2845] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:39:55 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:39:55.717549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be4826-63ab-49b1-a759-9c400ca7c1a5 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:55.728555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39be4826-63ab-49b1-a759-9c400ca7c1a5 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:55.785562 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e15561-68e6-4493-a0d9-2e63132542d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:55.786100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e15561-68e6-4493-a0d9-2e63132542d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:55.786385 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e15561-68e6-4493-a0d9-2e63132542d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:55.786665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e15561-68e6-4493-a0d9-2e63132542d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:55.787310 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e15561-68e6-4493-a0d9-2e63132542d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:55.789959 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e15561-68e6-4493-a0d9-2e63132542d9 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:55.906766 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1414/2846] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:39:55 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:39:55.910847 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bce199-a2ac-4963-b1b6-6703e5e9018b admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:55.921745 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bce199-a2ac-4963-b1b6-6703e5e9018b admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:55.926300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843c2f2-7378-4f07-9b38-254bb33cc939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:55.926890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843c2f2-7378-4f07-9b38-254bb33cc939 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:55.927169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843c2f2-7378-4f07-9b38-254bb33cc939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:55.927446 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8843c2f2-7378-4f07-9b38-254bb33cc939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:55.928119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843c2f2-7378-4f07-9b38-254bb33cc939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:55.936764 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8843c2f2-7378-4f07-9b38-254bb33cc939 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:55.939341 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1433/2847] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:39:55.943197 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e15561-68e6-4493-a0d9-2e63132542d9 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:55.953677 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e15561-68e6-4493-a0d9-2e63132542d9 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:55.958498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31f40e-add4-436f-8db9-d506dd71cd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:55.959114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f40e-add4-436f-8db9-d506dd71cd70 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:55.959390 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f40e-add4-436f-8db9-d506dd71cd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:55.959664 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31f40e-add4-436f-8db9-d506dd71cd70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:55.960898 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31f40e-add4-436f-8db9-d506dd71cd70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:55.962468 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31f40e-add4-436f-8db9-d506dd71cd70 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:55.963739 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:39:55.975060 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1415/2848] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:39:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:39:55.980307 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843c2f2-7378-4f07-9b38-254bb33cc939 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:55.990454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8843c2f2-7378-4f07-9b38-254bb33cc939 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:55.995203 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f848f9-fc67-41a4-817a-625f90cf8ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:55.995952 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f848f9-fc67-41a4-817a-625f90cf8ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:55.996239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f848f9-fc67-41a4-817a-625f90cf8ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:55.996521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f848f9-fc67-41a4-817a-625f90cf8ca1 admin admin] PATH_INFO: `/v3/projects/a2e61b47eb594b90a5f9078a0c7e9c96/users/fc5f05b0de434d349fa987c058979a70/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:56.011236 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f848f9-fc67-41a4-817a-625f90cf8ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e61b47eb594b90a5f9078a0c7e9c96, user_id=fc5f05b0de434d349fa987c058979a70, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:56.013959 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f848f9-fc67-41a4-817a-625f90cf8ca1 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:56.077283 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1434/2849] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:39:55 2022] PUT /identity/v3/projects/a2e61b47eb594b90a5f9078a0c7e9c96/users/fc5f05b0de434d349fa987c058979a70/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:39:56.087596 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1e359-8277-4185-bba8-5feaf563c522 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:56.087921 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1e359-8277-4185-bba8-5feaf563c522 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:56.088196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1e359-8277-4185-bba8-5feaf563c522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:56.116429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ced1e359-8277-4185-bba8-5feaf563c522 None None] MFA Rules not processed for user `fc5f05b0de434d349fa987c058979a70`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:39:56.117669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ced1e359-8277-4185-bba8-5feaf563c522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:56.180352 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1416/2850] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:39:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:39:56.184746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f848f9-fc67-41a4-817a-625f90cf8ca1 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:56.244279 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f848f9-fc67-41a4-817a-625f90cf8ca1 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:56.248809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b5899-56ab-44cd-8fef-9b83ac49f4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:56.249398 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b5899-56ab-44cd-8fef-9b83ac49f4ff admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:56.249673 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b5899-56ab-44cd-8fef-9b83ac49f4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:56.249941 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b5899-56ab-44cd-8fef-9b83ac49f4ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:56.250556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b5899-56ab-44cd-8fef-9b83ac49f4ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:56.252465 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8b5899-56ab-44cd-8fef-9b83ac49f4ff admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:56.386242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1435/2851] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:39:56 2022] POST /identity/v3/projects => generated 387 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:39:56.390306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1e359-8277-4185-bba8-5feaf563c522 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:56.401571 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1e359-8277-4185-bba8-5feaf563c522 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:56.457201 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125badb4-6cec-4c70-ac80-7d87a0a005dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:56.457844 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125badb4-6cec-4c70-ac80-7d87a0a005dc admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:56.458063 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125badb4-6cec-4c70-ac80-7d87a0a005dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:56.458334 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125badb4-6cec-4c70-ac80-7d87a0a005dc admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:56.458968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125badb4-6cec-4c70-ac80-7d87a0a005dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:56.460931 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125badb4-6cec-4c70-ac80-7d87a0a005dc admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:56.517315 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1417/2852] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:39:56 2022] POST /identity/v3/users => generated 294 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:39:56.521241 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b5899-56ab-44cd-8fef-9b83ac49f4ff admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:56.531669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b5899-56ab-44cd-8fef-9b83ac49f4ff admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:56.536316 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142c812-0fe5-4c4d-98e8-76f983a39786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:56.536907 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c812-0fe5-4c4d-98e8-76f983a39786 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:56.537181 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c812-0fe5-4c4d-98e8-76f983a39786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:56.537450 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142c812-0fe5-4c4d-98e8-76f983a39786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:56.538106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142c812-0fe5-4c4d-98e8-76f983a39786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:56.539679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142c812-0fe5-4c4d-98e8-76f983a39786 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:56.549973 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1436/2853] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:39:56.554430 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125badb4-6cec-4c70-ac80-7d87a0a005dc admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:56.564288 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125badb4-6cec-4c70-ac80-7d87a0a005dc admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:56.569126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2843eaaf-d08f-4d5e-8a32-087ba5c92463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:56.570623 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843eaaf-d08f-4d5e-8a32-087ba5c92463 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:56.570902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843eaaf-d08f-4d5e-8a32-087ba5c92463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:56.571178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2843eaaf-d08f-4d5e-8a32-087ba5c92463 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:56.571853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843eaaf-d08f-4d5e-8a32-087ba5c92463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:56.573434 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2843eaaf-d08f-4d5e-8a32-087ba5c92463 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:56.583355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1418/2854] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:39:56.587656 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142c812-0fe5-4c4d-98e8-76f983a39786 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:56.597727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142c812-0fe5-4c4d-98e8-76f983a39786 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:56.602344 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26268a5f-9017-4e57-a73c-54ef4eeb0925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:56.602995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26268a5f-9017-4e57-a73c-54ef4eeb0925 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:56.603330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26268a5f-9017-4e57-a73c-54ef4eeb0925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:56.603548 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26268a5f-9017-4e57-a73c-54ef4eeb0925 admin admin] PATH_INFO: `/v3/projects/d38013816d944e3b8565c9ca44693077/users/8f33017408fa441c962e6dbb46199634/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:56.618483 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26268a5f-9017-4e57-a73c-54ef4eeb0925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d38013816d944e3b8565c9ca44693077, user_id=8f33017408fa441c962e6dbb46199634, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:56.621153 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26268a5f-9017-4e57-a73c-54ef4eeb0925 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:56.689421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1437/2855] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:39:56 2022] PUT /identity/v3/projects/d38013816d944e3b8565c9ca44693077/users/8f33017408fa441c962e6dbb46199634/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:39:56.701521 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71ebb3-5f7f-433a-9817-84366b63f3b8 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:56.701806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71ebb3-5f7f-433a-9817-84366b63f3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:56.702082 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e71ebb3-5f7f-433a-9817-84366b63f3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:56.729944 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-6e71ebb3-5f7f-433a-9817-84366b63f3b8 None None] MFA Rules not processed for user `8f33017408fa441c962e6dbb46199634`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:39:56.731155 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-6e71ebb3-5f7f-433a-9817-84366b63f3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:56.785703 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:39:56.795619 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1419/2856] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:39:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:39:56.800189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26268a5f-9017-4e57-a73c-54ef4eeb0925 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:56.818106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:39:56.862223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26268a5f-9017-4e57-a73c-54ef4eeb0925 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:56.867577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280b8b4-90d5-44a8-bcc0-78faadefeafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:56.868209 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b8b4-90d5-44a8-bcc0-78faadefeafb admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:56.868492 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b8b4-90d5-44a8-bcc0-78faadefeafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:56.868766 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280b8b4-90d5-44a8-bcc0-78faadefeafb admin admin] PATH_INFO: `/v3/projects` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:56.869399 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280b8b4-90d5-44a8-bcc0-78faadefeafb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:56.871382 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280b8b4-90d5-44a8-bcc0-78faadefeafb admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:56.953585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1438/2857] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:39:56 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:39:56.957570 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71ebb3-5f7f-433a-9817-84366b63f3b8 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:56.968274 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e71ebb3-5f7f-433a-9817-84366b63f3b8 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:57.024590 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06d98e-91fe-40eb-8d32-dc25037baae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:57.025204 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06d98e-91fe-40eb-8d32-dc25037baae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:57.025484 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06d98e-91fe-40eb-8d32-dc25037baae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:57.025756 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a06d98e-91fe-40eb-8d32-dc25037baae3 admin admin] PATH_INFO: `/v3/users` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:57.026392 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06d98e-91fe-40eb-8d32-dc25037baae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:57.028338 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a06d98e-91fe-40eb-8d32-dc25037baae3 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:57.146930 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1420/2858] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:39:56 2022] POST /identity/v3/users => generated 300 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:39:57.150968 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280b8b4-90d5-44a8-bcc0-78faadefeafb admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:57.161097 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280b8b4-90d5-44a8-bcc0-78faadefeafb admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:57.166106 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76442faf-b95f-462c-a109-5ff5b4fd58da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:57.166698 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76442faf-b95f-462c-a109-5ff5b4fd58da admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:57.166975 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76442faf-b95f-462c-a109-5ff5b4fd58da admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:57.167247 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76442faf-b95f-462c-a109-5ff5b4fd58da admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:57.167914 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76442faf-b95f-462c-a109-5ff5b4fd58da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:57.169493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76442faf-b95f-462c-a109-5ff5b4fd58da admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:57.180440 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1439/2859] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:39:57.184418 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06d98e-91fe-40eb-8d32-dc25037baae3 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:57.194621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a06d98e-91fe-40eb-8d32-dc25037baae3 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:57.199536 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ce2ac-92de-4a2f-acb9-fde7a1a4c672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:57.200218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ce2ac-92de-4a2f-acb9-fde7a1a4c672 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:57.200499 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ce2ac-92de-4a2f-acb9-fde7a1a4c672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:57.200772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67ce2ac-92de-4a2f-acb9-fde7a1a4c672 admin admin] PATH_INFO: `/v3/projects/1cd75d016c62471496161d0462686e3b/users/37ee4f79d93f4c6a86cf2e768389917b/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:57.216066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ce2ac-92de-4a2f-acb9-fde7a1a4c672 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd75d016c62471496161d0462686e3b, user_id=37ee4f79d93f4c6a86cf2e768389917b, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:57.218649 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67ce2ac-92de-4a2f-acb9-fde7a1a4c672 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:57.306819 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1421/2860] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:39:57 2022] PUT /identity/v3/projects/1cd75d016c62471496161d0462686e3b/users/37ee4f79d93f4c6a86cf2e768389917b/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:39:57.310759 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76442faf-b95f-462c-a109-5ff5b4fd58da admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:57.320992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76442faf-b95f-462c-a109-5ff5b4fd58da admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:57.375649 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0849d-2538-4e8d-9d9b-1ce5012157ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:57.376271 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0849d-2538-4e8d-9d9b-1ce5012157ab admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:57.376551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0849d-2538-4e8d-9d9b-1ce5012157ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:57.376826 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0849d-2538-4e8d-9d9b-1ce5012157ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:57.377456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0849d-2538-4e8d-9d9b-1ce5012157ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:57.379105 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0849d-2538-4e8d-9d9b-1ce5012157ab admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:57.388712 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1440/2861] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:39:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:39:57.392834 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ce2ac-92de-4a2f-acb9-fde7a1a4c672 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:57.403413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67ce2ac-92de-4a2f-acb9-fde7a1a4c672 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:57.407924 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c084318-260a-4671-841d-0e83ab511678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:57.408560 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084318-260a-4671-841d-0e83ab511678 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:57.408839 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084318-260a-4671-841d-0e83ab511678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:57.409112 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084318-260a-4671-841d-0e83ab511678 admin admin] PATH_INFO: `/v3/projects/1cd75d016c62471496161d0462686e3b/users/37ee4f79d93f4c6a86cf2e768389917b/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:57.411011 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c084318-260a-4671-841d-0e83ab511678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd75d016c62471496161d0462686e3b, user_id=37ee4f79d93f4c6a86cf2e768389917b, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:57.414269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c084318-260a-4671-841d-0e83ab511678 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:57.424161 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1422/2862] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:39:57 2022] PUT /identity/v3/projects/1cd75d016c62471496161d0462686e3b/users/37ee4f79d93f4c6a86cf2e768389917b/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:39:57.434035 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0969d-8150-4b98-843f-2edd0da4b3d5 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:57.434344 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0969d-8150-4b98-843f-2edd0da4b3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:57.434615 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0969d-8150-4b98-843f-2edd0da4b3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:57.462056 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-2cb0969d-8150-4b98-843f-2edd0da4b3d5 None None] MFA Rules not processed for user `37ee4f79d93f4c6a86cf2e768389917b`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:39:57.463248 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2cb0969d-8150-4b98-843f-2edd0da4b3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:57.530862 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1441/2863] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:39:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:39:57.542043 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c084318-260a-4671-841d-0e83ab511678 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:57.543856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3c084318-260a-4671-841d-0e83ab511678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:57.600405 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c084318-260a-4671-841d-0e83ab511678 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:57.604037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71c403-bbeb-475e-9f79-f1b4f63848f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:57.604622 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71c403-bbeb-475e-9f79-f1b4f63848f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:57.604897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71c403-bbeb-475e-9f79-f1b4f63848f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:57.605167 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71c403-bbeb-475e-9f79-f1b4f63848f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:57.613803 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71c403-bbeb-475e-9f79-f1b4f63848f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:57.615206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd71c403-bbeb-475e-9f79-f1b4f63848f6 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:57.624425 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1423/2864] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:39:57 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:39:58.647149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0969d-8150-4b98-843f-2edd0da4b3d5 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:39:58.649145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-2cb0969d-8150-4b98-843f-2edd0da4b3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:39:58.660805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0969d-8150-4b98-843f-2edd0da4b3d5 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:39:58.665816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bce40-6148-408a-9d35-ca7b669b4c1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:39:58.665816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bce40-6148-408a-9d35-ca7b669b4c1c service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:39:58.665816 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bce40-6148-408a-9d35-ca7b669b4c1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:39:58.666429 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bce40-6148-408a-9d35-ca7b669b4c1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:39:58.674502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:39:58.723595 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bce40-6148-408a-9d35-ca7b669b4c1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:39:58.725054 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bce40-6148-408a-9d35-ca7b669b4c1c service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:39:58.734366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1442/2865] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:39:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:03.539860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71c403-bbeb-475e-9f79-f1b4f63848f6 service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:03.548457 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71c403-bbeb-475e-9f79-f1b4f63848f6 service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:03.552205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4956f-f81a-4bda-bcf3-a8d99c788b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:03.552794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4956f-f81a-4bda-bcf3-a8d99c788b29 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:03.553073 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4956f-f81a-4bda-bcf3-a8d99c788b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:03.553346 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4956f-f81a-4bda-bcf3-a8d99c788b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:03.598676 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:03.610459 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4956f-f81a-4bda-bcf3-a8d99c788b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:03.611960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4956f-f81a-4bda-bcf3-a8d99c788b29 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:03.621049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1424/2866] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:07.502493 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bce40-6148-408a-9d35-ca7b669b4c1c service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:07.563217 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bce40-6148-408a-9d35-ca7b669b4c1c service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:07.567749 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:07.568401 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:07.568682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:07.568955 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] PATH_INFO: `/v3/users/fc5f05b0de434d349fa987c058979a70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:07.570064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc5f05b0de434d349fa987c058979a70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:07.572301 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:07.805006 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc5f05b0de434d349fa987c058979a70'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:07.911879 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc5f05b0de434d349fa987c058979a70'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:07.912502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc5f05b0de434d349fa987c058979a70'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:07.913102 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc5f05b0de434d349fa987c058979a70'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:07.920980 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc5f05b0de434d349fa987c058979a70'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:07.938319 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1443/2867] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:40:07 2022] DELETE /identity/v3/users/fc5f05b0de434d349fa987c058979a70 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:07.942878 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f96b4-a488-4fed-847d-b674722a8791 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:07.943356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f96b4-a488-4fed-847d-b674722a8791 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:07.943496 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f96b4-a488-4fed-847d-b674722a8791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:07.963107 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-a46f96b4-a488-4fed-847d-b674722a8791 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:07.964330 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a46f96b4-a488-4fed-847d-b674722a8791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:08.029493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1425/2868] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:08.039030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:08.042119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:08.073149 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:08.097611 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aed1f7d-db8a-4699-8fae-7ca2342dba92 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:08.101267 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbb146-e8b5-473d-a43b-c2a773e87610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:08.101866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbb146-e8b5-473d-a43b-c2a773e87610 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:08.102679 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbb146-e8b5-473d-a43b-c2a773e87610 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:08.102957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbb146-e8b5-473d-a43b-c2a773e87610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:08.111984 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbb146-e8b5-473d-a43b-c2a773e87610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:08.113767 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbb146-e8b5-473d-a43b-c2a773e87610 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:08.123632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1444/2869] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:08.518918 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f96b4-a488-4fed-847d-b674722a8791 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:08.520374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a46f96b4-a488-4fed-847d-b674722a8791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:08.532449 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f96b4-a488-4fed-847d-b674722a8791 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:08.537106 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:08.540098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:08.540098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:08.540098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] PATH_INFO: `/v3/projects/a2e61b47eb594b90a5f9078a0c7e9c96` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:08.540098 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e61b47eb594b90a5f9078a0c7e9c96)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:08.541361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:08.672206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e61b47eb594b90a5f9078a0c7e9c96', 'request_id': 'req-d60e3981-9fc2-428e-bdbd-f03b9780447b'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:08.740473 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e61b47eb594b90a5f9078a0c7e9c96', 'request_id': 'req-d60e3981-9fc2-428e-bdbd-f03b9780447b'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:08.747706 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e61b47eb594b90a5f9078a0c7e9c96', 'request_id': 'req-d60e3981-9fc2-428e-bdbd-f03b9780447b'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:08.758066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] The token cache is being invalidate because project a2e61b47eb594b90a5f9078a0c7e9c96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:40:08.758942 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:08.759586 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:08.761110 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1426/2870] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:40:08 2022] DELETE /identity/v3/projects/a2e61b47eb594b90a5f9078a0c7e9c96 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:08.765264 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbb146-e8b5-473d-a43b-c2a773e87610 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:08.775258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbb146-e8b5-473d-a43b-c2a773e87610 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:08.831008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:08.831632 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:08.831946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:08.832226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] PATH_INFO: `/v3/users/8f33017408fa441c962e6dbb46199634` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:08.833343 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f33017408fa441c962e6dbb46199634)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:08.835257 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:09.022421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f33017408fa441c962e6dbb46199634'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:09.093230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f33017408fa441c962e6dbb46199634'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:09.093845 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f33017408fa441c962e6dbb46199634'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:09.094454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f33017408fa441c962e6dbb46199634'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:09.101530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f33017408fa441c962e6dbb46199634'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:09.119732 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1445/2871] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:40:08 2022] DELETE /identity/v3/users/8f33017408fa441c962e6dbb46199634 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:09.504004 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:09.505435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:09.519372 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e3981-9fc2-428e-bdbd-f03b9780447b admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:09.576893 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:09.577524 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:09.577803 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:09.578085 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] PATH_INFO: `/v3/projects/d38013816d944e3b8565c9ca44693077` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:09.579190 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d38013816d944e3b8565c9ca44693077)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:09.581118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:09.592695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:09.694890 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd38013816d944e3b8565c9ca44693077', 'request_id': 'req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:09.757598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd38013816d944e3b8565c9ca44693077', 'request_id': 'req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:09.764605 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd38013816d944e3b8565c9ca44693077', 'request_id': 'req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:09.775816 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] The token cache is being invalidate because project d38013816d944e3b8565c9ca44693077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:40:09.776114 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:09.776755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:09.778326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1427/2872] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:40:09 2022] DELETE /identity/v3/projects/d38013816d944e3b8565c9ca44693077 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:09.782373 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:09.783856 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:09.797195 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221bbed-ac44-4fb6-a672-eb456bedbd1a admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:09.848298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:09.858557 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159030ee-4ad0-42c9-a5cd-560faa337a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:09.859185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159030ee-4ad0-42c9-a5cd-560faa337a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:09.859461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159030ee-4ad0-42c9-a5cd-560faa337a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:09.859735 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159030ee-4ad0-42c9-a5cd-560faa337a5d admin admin] PATH_INFO: `/v3/users/37ee4f79d93f4c6a86cf2e768389917b` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:09.860876 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159030ee-4ad0-42c9-a5cd-560faa337a5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=37ee4f79d93f4c6a86cf2e768389917b)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:09.862782 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159030ee-4ad0-42c9-a5cd-560faa337a5d admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:10.042309 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-159030ee-4ad0-42c9-a5cd-560faa337a5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37ee4f79d93f4c6a86cf2e768389917b'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:10.122275 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-159030ee-4ad0-42c9-a5cd-560faa337a5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37ee4f79d93f4c6a86cf2e768389917b'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:10.122890 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-159030ee-4ad0-42c9-a5cd-560faa337a5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37ee4f79d93f4c6a86cf2e768389917b'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:10.123490 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-159030ee-4ad0-42c9-a5cd-560faa337a5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37ee4f79d93f4c6a86cf2e768389917b'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:10.131650 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-159030ee-4ad0-42c9-a5cd-560faa337a5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37ee4f79d93f4c6a86cf2e768389917b'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:10.150684 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1446/2873] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:40:09 2022] DELETE /identity/v3/users/37ee4f79d93f4c6a86cf2e768389917b => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:10.571284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:10.578653 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:10.585568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d7c4a-e74d-4207-9a52-6cf1bc98f59a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:10.642647 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:10.643275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:10.643555 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:10.643872 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] PATH_INFO: `/v3/projects/1cd75d016c62471496161d0462686e3b` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:10.644991 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd75d016c62471496161d0462686e3b)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:10.646895 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:10.741531 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd75d016c62471496161d0462686e3b', 'request_id': 'req-2c831393-9fdd-471e-bf15-9b72d4ed43fb'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:10.803517 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd75d016c62471496161d0462686e3b', 'request_id': 'req-2c831393-9fdd-471e-bf15-9b72d4ed43fb'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:10.810402 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd75d016c62471496161d0462686e3b', 'request_id': 'req-2c831393-9fdd-471e-bf15-9b72d4ed43fb'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:10.821108 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] The token cache is being invalidate because project 1cd75d016c62471496161d0462686e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:40:10.821397 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:10.822022 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-2c831393-9fdd-471e-bf15-9b72d4ed43fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:10.823505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1428/2874] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:40:10 2022] DELETE /identity/v3/projects/1cd75d016c62471496161d0462686e3b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:10.829455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e02fb-d398-4154-9cad-23e47eb60f73 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:10.829945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e02fb-d398-4154-9cad-23e47eb60f73 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:10.829945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e02fb-d398-4154-9cad-23e47eb60f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:10.850047 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-537e02fb-d398-4154-9cad-23e47eb60f73 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:10.851258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-537e02fb-d398-4154-9cad-23e47eb60f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:10.909667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1447/2875] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:10.918907 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc25a8d-25d1-403c-b623-8f849343c822 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:10.919220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc25a8d-25d1-403c-b623-8f849343c822 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:10.919498 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc25a8d-25d1-403c-b623-8f849343c822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:10.938972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-ebc25a8d-25d1-403c-b623-8f849343c822 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:10.940208 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-ebc25a8d-25d1-403c-b623-8f849343c822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:10.948314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1429/2876] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:10.952456 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e02fb-d398-4154-9cad-23e47eb60f73 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:10.962565 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e02fb-d398-4154-9cad-23e47eb60f73 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:10.967044 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b6f9f-6445-4735-b4ec-d7f2d5e994d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:10.967641 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6f9f-6445-4735-b4ec-d7f2d5e994d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:10.967953 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6f9f-6445-4735-b4ec-d7f2d5e994d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:10.968230 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659b6f9f-6445-4735-b4ec-d7f2d5e994d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:10.968866 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b6f9f-6445-4735-b4ec-d7f2d5e994d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:40:10.969171 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b6f9f-6445-4735-b4ec-d7f2d5e994d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:10.970750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659b6f9f-6445-4735-b4ec-d7f2d5e994d4 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:10.982561 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1448/2877] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:40:10.986852 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc25a8d-25d1-403c-b623-8f849343c822 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:10.996532 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc25a8d-25d1-403c-b623-8f849343c822 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:11.001217 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83868d-1ddf-4c23-8e34-26e9333a714a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:11.001811 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83868d-1ddf-4c23-8e34-26e9333a714a admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:11.002088 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83868d-1ddf-4c23-8e34-26e9333a714a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:11.002387 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83868d-1ddf-4c23-8e34-26e9333a714a admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:11.003038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83868d-1ddf-4c23-8e34-26e9333a714a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:11.004968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de83868d-1ddf-4c23-8e34-26e9333a714a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:11.088696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1430/2878] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:40:10 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:40:11.092837 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b6f9f-6445-4735-b4ec-d7f2d5e994d4 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:11.103815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659b6f9f-6445-4735-b4ec-d7f2d5e994d4 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:11.153250 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:11.158884 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45323327-814d-4016-aa81-4dfdaab35cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:11.159479 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323327-814d-4016-aa81-4dfdaab35cae admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:11.159777 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323327-814d-4016-aa81-4dfdaab35cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:11.160064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45323327-814d-4016-aa81-4dfdaab35cae admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:11.160683 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45323327-814d-4016-aa81-4dfdaab35cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:11.162585 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45323327-814d-4016-aa81-4dfdaab35cae admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:11.265646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1449/2879] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:40:11 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:40:11.269692 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83868d-1ddf-4c23-8e34-26e9333a714a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:11.275957 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:11.281286 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de83868d-1ddf-4c23-8e34-26e9333a714a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:11.285967 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f26aa8-9ada-4806-ac76-b8a159654ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:11.286558 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f26aa8-9ada-4806-ac76-b8a159654ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:11.286836 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f26aa8-9ada-4806-ac76-b8a159654ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:11.287192 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f26aa8-9ada-4806-ac76-b8a159654ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:11.287860 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f26aa8-9ada-4806-ac76-b8a159654ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:11.290023 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f26aa8-9ada-4806-ac76-b8a159654ce1 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:11.299960 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1431/2880] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:40:11.304030 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45323327-814d-4016-aa81-4dfdaab35cae admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:11.314726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45323327-814d-4016-aa81-4dfdaab35cae admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:11.319467 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87134a0b-e2a3-4dd6-8051-8b7b2a0204dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:11.320095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87134a0b-e2a3-4dd6-8051-8b7b2a0204dc admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:11.320657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87134a0b-e2a3-4dd6-8051-8b7b2a0204dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:11.321424 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87134a0b-e2a3-4dd6-8051-8b7b2a0204dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:11.322114 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87134a0b-e2a3-4dd6-8051-8b7b2a0204dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:11.324227 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87134a0b-e2a3-4dd6-8051-8b7b2a0204dc admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:11.335130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1450/2881] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:40:11.339806 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f26aa8-9ada-4806-ac76-b8a159654ce1 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:11.350291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f26aa8-9ada-4806-ac76-b8a159654ce1 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:11.355049 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a1f45-de15-4222-afad-a59f744deb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:11.355699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a1f45-de15-4222-afad-a59f744deb40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:11.356012 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a1f45-de15-4222-afad-a59f744deb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:11.356291 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a1f45-de15-4222-afad-a59f744deb40 admin admin] PATH_INFO: `/v3/projects/d9f806c95ab94950b06e72d5a393924f/users/85d6cc3c43254a49bcdc895d15b4de13/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:11.371322 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a1f45-de15-4222-afad-a59f744deb40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9f806c95ab94950b06e72d5a393924f, user_id=85d6cc3c43254a49bcdc895d15b4de13, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:11.373985 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a1f45-de15-4222-afad-a59f744deb40 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:11.425556 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1432/2882] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:40:11 2022] PUT /identity/v3/projects/d9f806c95ab94950b06e72d5a393924f/users/85d6cc3c43254a49bcdc895d15b4de13/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:11.438408 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5320d6-dfe3-4fb6-8aa9-30f7df9be5bd None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:11.438691 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5320d6-dfe3-4fb6-8aa9-30f7df9be5bd None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:11.438986 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5320d6-dfe3-4fb6-8aa9-30f7df9be5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:11.468165 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-8f5320d6-dfe3-4fb6-8aa9-30f7df9be5bd None None] MFA Rules not processed for user `85d6cc3c43254a49bcdc895d15b4de13`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:11.469380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8f5320d6-dfe3-4fb6-8aa9-30f7df9be5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:11.533101 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1451/2883] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:11.537662 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a1f45-de15-4222-afad-a59f744deb40 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:11.599276 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a1f45-de15-4222-afad-a59f744deb40 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:11.603920 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3741ccf-985e-42dc-8fa9-896fda8576b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:11.604515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3741ccf-985e-42dc-8fa9-896fda8576b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:11.604800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3741ccf-985e-42dc-8fa9-896fda8576b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:11.605079 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3741ccf-985e-42dc-8fa9-896fda8576b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:11.605717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3741ccf-985e-42dc-8fa9-896fda8576b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:11.607607 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3741ccf-985e-42dc-8fa9-896fda8576b8 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:11.697132 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1433/2884] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:40:11 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:40:11.701143 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5320d6-dfe3-4fb6-8aa9-30f7df9be5bd None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:11.711838 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5320d6-dfe3-4fb6-8aa9-30f7df9be5bd None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:11.768692 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6666b4-87c5-49b9-871a-949ad4500fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:11.769300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6666b4-87c5-49b9-871a-949ad4500fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:11.769582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6666b4-87c5-49b9-871a-949ad4500fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:11.769853 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6666b4-87c5-49b9-871a-949ad4500fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:11.770488 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6666b4-87c5-49b9-871a-949ad4500fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:11.772594 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6666b4-87c5-49b9-871a-949ad4500fb3 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:11.854888 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1452/2885] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:40:11 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:40:11.858916 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3741ccf-985e-42dc-8fa9-896fda8576b8 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:11.870037 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3741ccf-985e-42dc-8fa9-896fda8576b8 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:11.875620 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c46ac-1408-40c0-b99a-f78439461ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:11.876258 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c46ac-1408-40c0-b99a-f78439461ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:11.876541 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c46ac-1408-40c0-b99a-f78439461ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:11.876818 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c46ac-1408-40c0-b99a-f78439461ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:11.877456 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c46ac-1408-40c0-b99a-f78439461ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:11.879066 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c46ac-1408-40c0-b99a-f78439461ca8 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:11.890384 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1434/2886] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:40:11.894362 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6666b4-87c5-49b9-871a-949ad4500fb3 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:11.905646 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6666b4-87c5-49b9-871a-949ad4500fb3 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:11.910521 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611c6b3-9fa2-4407-b441-81721e608efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:11.911117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611c6b3-9fa2-4407-b441-81721e608efd admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:11.911397 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611c6b3-9fa2-4407-b441-81721e608efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:11.911674 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611c6b3-9fa2-4407-b441-81721e608efd admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:11.912348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611c6b3-9fa2-4407-b441-81721e608efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:11.913935 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611c6b3-9fa2-4407-b441-81721e608efd admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:11.923809 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1453/2887] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:40:11.928123 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c46ac-1408-40c0-b99a-f78439461ca8 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:11.938180 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c46ac-1408-40c0-b99a-f78439461ca8 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:11.943019 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d361dbe-3fb6-48a1-a459-1212a5a912f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:11.943663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d361dbe-3fb6-48a1-a459-1212a5a912f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:11.943976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d361dbe-3fb6-48a1-a459-1212a5a912f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:11.944285 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d361dbe-3fb6-48a1-a459-1212a5a912f2 admin admin] PATH_INFO: `/v3/projects/c1f6d5e84ad94eee956d7f4ec29f0839/users/b29a6633c7524d81b43d9ee916be9698/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:11.959650 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d361dbe-3fb6-48a1-a459-1212a5a912f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f6d5e84ad94eee956d7f4ec29f0839, user_id=b29a6633c7524d81b43d9ee916be9698, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:11.962948 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d361dbe-3fb6-48a1-a459-1212a5a912f2 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:12.037846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1435/2888] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:40:11 2022] PUT /identity/v3/projects/c1f6d5e84ad94eee956d7f4ec29f0839/users/b29a6633c7524d81b43d9ee916be9698/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:12.048442 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c949c-b0bf-4bbc-b432-73e634fb1ffd None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:12.048727 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c949c-b0bf-4bbc-b432-73e634fb1ffd None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:12.053095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c949c-b0bf-4bbc-b432-73e634fb1ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:12.077022 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-048c949c-b0bf-4bbc-b432-73e634fb1ffd None None] MFA Rules not processed for user `b29a6633c7524d81b43d9ee916be9698`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:12.078216 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-048c949c-b0bf-4bbc-b432-73e634fb1ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:12.109792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:12.140717 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1454/2889] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:12.145153 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d361dbe-3fb6-48a1-a459-1212a5a912f2 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:12.205897 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d361dbe-3fb6-48a1-a459-1212a5a912f2 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:12.210537 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375b619-58c3-4a8e-a83f-97de35f2acac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:12.211130 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b619-58c3-4a8e-a83f-97de35f2acac admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:12.211952 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b619-58c3-4a8e-a83f-97de35f2acac admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:12.212225 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375b619-58c3-4a8e-a83f-97de35f2acac admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:12.212845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375b619-58c3-4a8e-a83f-97de35f2acac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:12.214728 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2375b619-58c3-4a8e-a83f-97de35f2acac admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:12.290131 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1436/2890] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:40:12 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:40:12.294238 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c949c-b0bf-4bbc-b432-73e634fb1ffd None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:12.305700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c949c-b0bf-4bbc-b432-73e634fb1ffd None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:12.360049 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d44d3-d874-4a5d-8bee-171e489a6d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:12.360575 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d44d3-d874-4a5d-8bee-171e489a6d76 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:12.360851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d44d3-d874-4a5d-8bee-171e489a6d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:12.361117 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d44d3-d874-4a5d-8bee-171e489a6d76 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:12.361740 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d44d3-d874-4a5d-8bee-171e489a6d76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:12.363624 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d44d3-d874-4a5d-8bee-171e489a6d76 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:12.429741 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1455/2891] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:40:12 2022] POST /identity/v3/users => generated 296 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:40:12.434173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375b619-58c3-4a8e-a83f-97de35f2acac admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:12.439699 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:12.445126 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375b619-58c3-4a8e-a83f-97de35f2acac admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:12.450888 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c69828-d15c-45e7-9d1e-e4915c93c84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:12.451583 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69828-d15c-45e7-9d1e-e4915c93c84a admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:12.451894 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69828-d15c-45e7-9d1e-e4915c93c84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:12.452171 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69828-d15c-45e7-9d1e-e4915c93c84a admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:12.452805 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c69828-d15c-45e7-9d1e-e4915c93c84a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:12.454491 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c69828-d15c-45e7-9d1e-e4915c93c84a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:12.466277 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1437/2892] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:40:12.470352 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d44d3-d874-4a5d-8bee-171e489a6d76 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:12.480980 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d44d3-d874-4a5d-8bee-171e489a6d76 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:12.485868 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c12c-cd03-4101-9d58-9a79e37b24d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:12.486522 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c12c-cd03-4101-9d58-9a79e37b24d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:12.486805 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c12c-cd03-4101-9d58-9a79e37b24d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:12.487083 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3c12c-cd03-4101-9d58-9a79e37b24d7 admin admin] PATH_INFO: `/v3/projects/1a660e385d8e4ab3b030590a43126ba1/users/8139d83ff34c43a7b1eba4b23aaba659/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:12.502296 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3c12c-cd03-4101-9d58-9a79e37b24d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a660e385d8e4ab3b030590a43126ba1, user_id=8139d83ff34c43a7b1eba4b23aaba659, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:12.504982 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd3c12c-cd03-4101-9d58-9a79e37b24d7 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:12.551924 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1456/2893] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:40:12 2022] PUT /identity/v3/projects/1a660e385d8e4ab3b030590a43126ba1/users/8139d83ff34c43a7b1eba4b23aaba659/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:12.555794 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c69828-d15c-45e7-9d1e-e4915c93c84a admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:12.566440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c69828-d15c-45e7-9d1e-e4915c93c84a admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:12.622053 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35658030-ab51-4133-8a31-efed97c5fb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:12.622694 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35658030-ab51-4133-8a31-efed97c5fb99 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:12.622853 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35658030-ab51-4133-8a31-efed97c5fb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:12.623125 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35658030-ab51-4133-8a31-efed97c5fb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:12.623757 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35658030-ab51-4133-8a31-efed97c5fb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:12.625355 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35658030-ab51-4133-8a31-efed97c5fb99 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:12.635342 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1438/2894] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:40:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:40:12.639370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c12c-cd03-4101-9d58-9a79e37b24d7 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:12.649477 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3c12c-cd03-4101-9d58-9a79e37b24d7 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:12.654616 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ed298-ed78-4ab8-a29c-de6761d8d9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:12.655256 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ed298-ed78-4ab8-a29c-de6761d8d9c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:12.655528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ed298-ed78-4ab8-a29c-de6761d8d9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:12.655830 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ed298-ed78-4ab8-a29c-de6761d8d9c5 admin admin] PATH_INFO: `/v3/projects/1a660e385d8e4ab3b030590a43126ba1/users/8139d83ff34c43a7b1eba4b23aaba659/roles/4f1f6a27bf0e4b7190e52bcebb14bf70` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:12.657726 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ed298-ed78-4ab8-a29c-de6761d8d9c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a660e385d8e4ab3b030590a43126ba1, user_id=8139d83ff34c43a7b1eba4b23aaba659, role_id=4f1f6a27bf0e4b7190e52bcebb14bf70)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:12.660363 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ed298-ed78-4ab8-a29c-de6761d8d9c5 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:12.670634 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1457/2895] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:40:12 2022] PUT /identity/v3/projects/1a660e385d8e4ab3b030590a43126ba1/users/8139d83ff34c43a7b1eba4b23aaba659/roles/4f1f6a27bf0e4b7190e52bcebb14bf70 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:12.682052 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1c5b1-a343-4439-b4a9-efb107bf6329 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:12.682413 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1c5b1-a343-4439-b4a9-efb107bf6329 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:12.682696 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1c5b1-a343-4439-b4a9-efb107bf6329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:12.709994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-8cc1c5b1-a343-4439-b4a9-efb107bf6329 None None] MFA Rules not processed for user `8139d83ff34c43a7b1eba4b23aaba659`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:12.711205 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8cc1c5b1-a343-4439-b4a9-efb107bf6329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:12.779856 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1439/2896] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:12.792021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ed298-ed78-4ab8-a29c-de6761d8d9c5 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:12.793778 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-754ed298-ed78-4ab8-a29c-de6761d8d9c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:12.850987 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ed298-ed78-4ab8-a29c-de6761d8d9c5 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:12.855218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e3e7-ac4a-45ea-bb46-f9e7f7f94ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:12.855835 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e3e7-ac4a-45ea-bb46-f9e7f7f94ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:12.856116 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e3e7-ac4a-45ea-bb46-f9e7f7f94ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:12.856387 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e3e7-ac4a-45ea-bb46-f9e7f7f94ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:12.865012 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6e3e7-ac4a-45ea-bb46-f9e7f7f94ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:12.866420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6e3e7-ac4a-45ea-bb46-f9e7f7f94ce1 service cinder] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:12.876306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1458/2897] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:13.000327 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1c5b1-a343-4439-b4a9-efb107bf6329 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:13.001825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-8cc1c5b1-a343-4439-b4a9-efb107bf6329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:13.013955 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1c5b1-a343-4439-b4a9-efb107bf6329 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:13.018732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe9a40-af14-4a6e-8d6f-51116e8e4509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:13.018732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9a40-af14-4a6e-8d6f-51116e8e4509 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:13.018732 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9a40-af14-4a6e-8d6f-51116e8e4509 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:13.019196 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe9a40-af14-4a6e-8d6f-51116e8e4509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:13.078624 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe9a40-af14-4a6e-8d6f-51116e8e4509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:13.080087 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe9a40-af14-4a6e-8d6f-51116e8e4509 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:13.089656 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1440/2898] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:17.228946 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e3e7-ac4a-45ea-bb46-f9e7f7f94ce1 service cinder] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:17.259454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:17.312897 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e3e7-ac4a-45ea-bb46-f9e7f7f94ce1 service cinder] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:17.317455 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:17.318075 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:17.318349 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:17.318619 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] PATH_INFO: `/v3/users/85d6cc3c43254a49bcdc895d15b4de13` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:17.319733 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d6cc3c43254a49bcdc895d15b4de13)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:17.321972 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:17.500384 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d6cc3c43254a49bcdc895d15b4de13'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:17.568905 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d6cc3c43254a49bcdc895d15b4de13'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:17.569471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d6cc3c43254a49bcdc895d15b4de13'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:17.570081 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d6cc3c43254a49bcdc895d15b4de13'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:17.577623 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d6cc3c43254a49bcdc895d15b4de13'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:17.595977 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1459/2899] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:40:17 2022] DELETE /identity/v3/users/85d6cc3c43254a49bcdc895d15b4de13 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:17.600409 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426633-ff62-451f-8301-596b28b87310 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:17.600945 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426633-ff62-451f-8301-596b28b87310 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:17.601057 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a426633-ff62-451f-8301-596b28b87310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:17.618284 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=100 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:17.630597 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-3a426633-ff62-451f-8301-596b28b87310 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:17.631833 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3a426633-ff62-451f-8301-596b28b87310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:17.690071 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1441/2900] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:17.699895 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:17.701297 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:17.757388 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d02b4d-c5f2-4764-aca3-e787c4e9b871 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:17.761530 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269758a-0162-49a7-a86d-e2b5cf138ca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:17.762130 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269758a-0162-49a7-a86d-e2b5cf138ca7 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:17.762409 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269758a-0162-49a7-a86d-e2b5cf138ca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:17.762682 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269758a-0162-49a7-a86d-e2b5cf138ca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:17.771383 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269758a-0162-49a7-a86d-e2b5cf138ca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:17.772846 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269758a-0162-49a7-a86d-e2b5cf138ca7 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:17.782769 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1460/2901] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:18.198502 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a426633-ff62-451f-8301-596b28b87310 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:18.199984 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-3a426633-ff62-451f-8301-596b28b87310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:18.212636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a426633-ff62-451f-8301-596b28b87310 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:18.217228 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:18.217861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:18.218138 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:18.218408 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] PATH_INFO: `/v3/projects/d9f806c95ab94950b06e72d5a393924f` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:18.219542 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9f806c95ab94950b06e72d5a393924f)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:18.221478 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:18.346089 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9f806c95ab94950b06e72d5a393924f', 'request_id': 'req-a8e9fa0b-5735-43a4-9a86-d0779c68c430'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:18.414245 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9f806c95ab94950b06e72d5a393924f', 'request_id': 'req-a8e9fa0b-5735-43a4-9a86-d0779c68c430'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:18.420455 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9f806c95ab94950b06e72d5a393924f', 'request_id': 'req-a8e9fa0b-5735-43a4-9a86-d0779c68c430'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:18.430630 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] The token cache is being invalidate because project d9f806c95ab94950b06e72d5a393924f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:40:18.430915 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:18.432075 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:18.435642 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1442/2902] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:40:18 2022] DELETE /identity/v3/projects/d9f806c95ab94950b06e72d5a393924f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:18.437588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269758a-0162-49a7-a86d-e2b5cf138ca7 service neutron] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:18.448088 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269758a-0162-49a7-a86d-e2b5cf138ca7 service neutron] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:18.470742 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:18.504551 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:18.505243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:18.505526 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:18.505820 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] PATH_INFO: `/v3/users/b29a6633c7524d81b43d9ee916be9698` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:18.506920 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b29a6633c7524d81b43d9ee916be9698)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:18.508849 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:18.683579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b29a6633c7524d81b43d9ee916be9698'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:18.763185 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b29a6633c7524d81b43d9ee916be9698'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:18.763731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b29a6633c7524d81b43d9ee916be9698'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:18.764367 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b29a6633c7524d81b43d9ee916be9698'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:18.771825 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b29a6633c7524d81b43d9ee916be9698'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:18.793528 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1461/2903] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:40:18 2022] DELETE /identity/v3/users/b29a6633c7524d81b43d9ee916be9698 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:19.456226 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:19.461997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:19.471326 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9fa0b-5735-43a4-9a86-d0779c68c430 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:19.528069 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:19.528626 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:19.528902 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:19.529218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] PATH_INFO: `/v3/projects/c1f6d5e84ad94eee956d7f4ec29f0839` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:19.530301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f6d5e84ad94eee956d7f4ec29f0839)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:19.532254 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:19.622589 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:19.637479 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f6d5e84ad94eee956d7f4ec29f0839', 'request_id': 'req-def376fe-c951-46d2-ac38-a33c88862606'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:19.698720 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f6d5e84ad94eee956d7f4ec29f0839', 'request_id': 'req-def376fe-c951-46d2-ac38-a33c88862606'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:19.705306 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f6d5e84ad94eee956d7f4ec29f0839', 'request_id': 'req-def376fe-c951-46d2-ac38-a33c88862606'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:19.716540 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] The token cache is being invalidate because project c1f6d5e84ad94eee956d7f4ec29f0839 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:40:19.716829 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:19.717471 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:19.718971 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1443/2904] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:40:19 2022] DELETE /identity/v3/projects/c1f6d5e84ad94eee956d7f4ec29f0839 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:19.723173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:19.729957 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:19.744738 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb0fe1-2df2-448b-bcb4-98da052c8a82 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:19.800570 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8cc43-c4bb-4833-b2ce-43b1e5533345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:19.801145 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8cc43-c4bb-4833-b2ce-43b1e5533345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:19.801406 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8cc43-c4bb-4833-b2ce-43b1e5533345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:19.801681 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8cc43-c4bb-4833-b2ce-43b1e5533345 admin admin] PATH_INFO: `/v3/users/8139d83ff34c43a7b1eba4b23aaba659` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:19.802925 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8cc43-c4bb-4833-b2ce-43b1e5533345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8139d83ff34c43a7b1eba4b23aaba659)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:19.804917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8cc43-c4bb-4833-b2ce-43b1e5533345 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:19.975669 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d2a8cc43-c4bb-4833-b2ce-43b1e5533345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8139d83ff34c43a7b1eba4b23aaba659'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:19.977330 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:20.043064 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d2a8cc43-c4bb-4833-b2ce-43b1e5533345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8139d83ff34c43a7b1eba4b23aaba659'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:20.043686 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d2a8cc43-c4bb-4833-b2ce-43b1e5533345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8139d83ff34c43a7b1eba4b23aaba659'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:20.044317 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d2a8cc43-c4bb-4833-b2ce-43b1e5533345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8139d83ff34c43a7b1eba4b23aaba659'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:20.052324 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-d2a8cc43-c4bb-4833-b2ce-43b1e5533345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8139d83ff34c43a7b1eba4b23aaba659'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:20.070500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1462/2905] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:40:19 2022] DELETE /identity/v3/users/8139d83ff34c43a7b1eba4b23aaba659 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:40:20.477194 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:20.479301 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:20.493861 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def376fe-c951-46d2-ac38-a33c88862606 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:20.550370 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:20.551005 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:20.551335 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:20.551618 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] PATH_INFO: `/v3/projects/1a660e385d8e4ab3b030590a43126ba1` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:20.552790 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a660e385d8e4ab3b030590a43126ba1)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:20.557084 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:20.659772 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a660e385d8e4ab3b030590a43126ba1', 'request_id': 'req-76c93737-7f2e-437c-903c-b750c25cd47a'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:20.728923 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a660e385d8e4ab3b030590a43126ba1', 'request_id': 'req-76c93737-7f2e-437c-903c-b750c25cd47a'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:20.735316 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a660e385d8e4ab3b030590a43126ba1', 'request_id': 'req-76c93737-7f2e-437c-903c-b750c25cd47a'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:20.746508 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] The token cache is being invalidate because project 1a660e385d8e4ab3b030590a43126ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:40:20.746848 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:20.747493 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-76c93737-7f2e-437c-903c-b750c25cd47a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:20.749030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1444/2906] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:40:20 2022] DELETE /identity/v3/projects/1a660e385d8e4ab3b030590a43126ba1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:20.765098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a9f90-5523-4adc-936e-1746f5db8ca3 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:20.765906 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a9f90-5523-4adc-936e-1746f5db8ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:20.766104 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a9f90-5523-4adc-936e-1746f5db8ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:20.795949 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-961a9f90-5523-4adc-936e-1746f5db8ca3 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:20.797151 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-961a9f90-5523-4adc-936e-1746f5db8ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:20.856706 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1463/2907] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:20.865096 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897d69a-086a-475c-a836-650988908cbc None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:20.865448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897d69a-086a-475c-a836-650988908cbc None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:20.865665 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897d69a-086a-475c-a836-650988908cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:20.897976 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7897d69a-086a-475c-a836-650988908cbc None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:20.899179 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7897d69a-086a-475c-a836-650988908cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:20.925880 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1445/2908] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:20.929996 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a9f90-5523-4adc-936e-1746f5db8ca3 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:20.957461 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a9f90-5523-4adc-936e-1746f5db8ca3 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:20.962004 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffcc5b-4910-4706-a065-9321f02b7355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:20.963226 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffcc5b-4910-4706-a065-9321f02b7355 admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:20.963502 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffcc5b-4910-4706-a065-9321f02b7355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:20.963788 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffcc5b-4910-4706-a065-9321f02b7355 admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:20.964422 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffcc5b-4910-4706-a065-9321f02b7355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:40:20.964723 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffcc5b-4910-4706-a065-9321f02b7355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:20.966421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffcc5b-4910-4706-a065-9321f02b7355 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:20.977562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1464/2909] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:40:20.982314 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897d69a-086a-475c-a836-650988908cbc None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:20.991538 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897d69a-086a-475c-a836-650988908cbc None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:20.999517 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4285743-a1d0-463e-9fae-7f55220b8b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:21.000178 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4285743-a1d0-463e-9fae-7f55220b8b95 admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:21.000453 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4285743-a1d0-463e-9fae-7f55220b8b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:21.000724 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4285743-a1d0-463e-9fae-7f55220b8b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:21.001367 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4285743-a1d0-463e-9fae-7f55220b8b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:21.003257 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4285743-a1d0-463e-9fae-7f55220b8b95 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:21.014185 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:21.113835 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1446/2910] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:40:20 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:40:21.118001 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffcc5b-4910-4706-a065-9321f02b7355 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:21.128562 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffcc5b-4910-4706-a065-9321f02b7355 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:21.184299 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189680cc-dab7-4174-868b-da0973038dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:21.184861 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189680cc-dab7-4174-868b-da0973038dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:21.185139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189680cc-dab7-4174-868b-da0973038dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:21.185412 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189680cc-dab7-4174-868b-da0973038dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:21.186034 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189680cc-dab7-4174-868b-da0973038dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:21.188067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189680cc-dab7-4174-868b-da0973038dc7 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:21.266601 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1465/2911] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:40:21 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:40:21.270598 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4285743-a1d0-463e-9fae-7f55220b8b95 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:21.280774 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4285743-a1d0-463e-9fae-7f55220b8b95 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:21.285351 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237105b-35a4-4d51-a2a8-db0e485ebe85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:21.285940 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237105b-35a4-4d51-a2a8-db0e485ebe85 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:21.286218 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237105b-35a4-4d51-a2a8-db0e485ebe85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:21.286491 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237105b-35a4-4d51-a2a8-db0e485ebe85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:21.287118 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237105b-35a4-4d51-a2a8-db0e485ebe85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:21.288731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c237105b-35a4-4d51-a2a8-db0e485ebe85 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:21.298419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1447/2912] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:40:21.302109 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189680cc-dab7-4174-868b-da0973038dc7 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:21.312250 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189680cc-dab7-4174-868b-da0973038dc7 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:21.316811 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c57d5b-48f0-4146-95e1-43fa00a53cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:21.317394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c57d5b-48f0-4146-95e1-43fa00a53cca admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:21.317672 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c57d5b-48f0-4146-95e1-43fa00a53cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:21.317945 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c57d5b-48f0-4146-95e1-43fa00a53cca admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:21.318574 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c57d5b-48f0-4146-95e1-43fa00a53cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:21.320169 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c57d5b-48f0-4146-95e1-43fa00a53cca admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:21.331113 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1466/2913] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:40:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:40:21.335140 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237105b-35a4-4d51-a2a8-db0e485ebe85 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:21.345021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237105b-35a4-4d51-a2a8-db0e485ebe85 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:21.349594 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ab023-b247-4a26-ac2f-ece3c8e27384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:21.350225 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ab023-b247-4a26-ac2f-ece3c8e27384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:21.350505 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ab023-b247-4a26-ac2f-ece3c8e27384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:21.350808 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ab023-b247-4a26-ac2f-ece3c8e27384 admin admin] PATH_INFO: `/v3/projects/500e1fa77cb6492381d8b94dfda93b0f/users/1b89d153663a456b844eba90ca52ccb6/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:21.365038 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ab023-b247-4a26-ac2f-ece3c8e27384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=500e1fa77cb6492381d8b94dfda93b0f, user_id=1b89d153663a456b844eba90ca52ccb6, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:21.367659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ab023-b247-4a26-ac2f-ece3c8e27384 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:21.437731 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1448/2914] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:40:21 2022] PUT /identity/v3/projects/500e1fa77cb6492381d8b94dfda93b0f/users/1b89d153663a456b844eba90ca52ccb6/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:21.449625 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa18d7-d077-4952-bedd-edc35978b3ab None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:21.449908 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa18d7-d077-4952-bedd-edc35978b3ab None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:21.450184 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa18d7-d077-4952-bedd-edc35978b3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:21.463159 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:21.477119 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-79aa18d7-d077-4952-bedd-edc35978b3ab None None] MFA Rules not processed for user `1b89d153663a456b844eba90ca52ccb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:21.479675 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-79aa18d7-d077-4952-bedd-edc35978b3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:21.539454 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1467/2915] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:21.551013 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ab023-b247-4a26-ac2f-ece3c8e27384 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:21.553248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-e97ab023-b247-4a26-ac2f-ece3c8e27384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:21.625356 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ab023-b247-4a26-ac2f-ece3c8e27384 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:21.629026 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e330b5-f748-4106-805a-80e0c435fc84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:21.629627 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e330b5-f748-4106-805a-80e0c435fc84 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:21.629936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e330b5-f748-4106-805a-80e0c435fc84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:21.630230 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e330b5-f748-4106-805a-80e0c435fc84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:21.638490 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e330b5-f748-4106-805a-80e0c435fc84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:21.639983 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e330b5-f748-4106-805a-80e0c435fc84 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:21.648722 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1449/2916] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:31.520586 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa18d7-d077-4952-bedd-edc35978b3ab None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:31.584851 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa18d7-d077-4952-bedd-edc35978b3ab None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:31.589629 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:31.590258 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:31.591070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:31.591346 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] PATH_INFO: `/v3/users/1b89d153663a456b844eba90ca52ccb6` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:31.592507 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b89d153663a456b844eba90ca52ccb6)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:31.594874 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:31.644695 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:31.824475 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b89d153663a456b844eba90ca52ccb6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:31.900366 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b89d153663a456b844eba90ca52ccb6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:31.900995 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b89d153663a456b844eba90ca52ccb6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:31.901659 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b89d153663a456b844eba90ca52ccb6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:31.910124 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b89d153663a456b844eba90ca52ccb6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:31.926828 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1468/2917] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:40:31 2022] DELETE /identity/v3/users/1b89d153663a456b844eba90ca52ccb6 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:31.931263 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e88833-f3ef-4c3b-83c2-20f2956ec0aa None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:31.932030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e88833-f3ef-4c3b-83c2-20f2956ec0aa None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:31.932030 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e88833-f3ef-4c3b-83c2-20f2956ec0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:31.960380 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-c2e88833-f3ef-4c3b-83c2-20f2956ec0aa None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:31.961641 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c2e88833-f3ef-4c3b-83c2-20f2956ec0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:32.026148 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1450/2918] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:32.036581 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:32.038193 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:32.058273 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:32.097541 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef56f3a-56d2-4191-bc0b-1b55957e1b1f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:32.101073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea705ed-cf69-4625-8b37-0c1f9d20a28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:32.101775 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea705ed-cf69-4625-8b37-0c1f9d20a28e service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:32.102058 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea705ed-cf69-4625-8b37-0c1f9d20a28e service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:32.102335 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea705ed-cf69-4625-8b37-0c1f9d20a28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:32.111579 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea705ed-cf69-4625-8b37-0c1f9d20a28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:32.113554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea705ed-cf69-4625-8b37-0c1f9d20a28e service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:32.122829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1469/2919] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:32.524332 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e88833-f3ef-4c3b-83c2-20f2956ec0aa None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:32.525754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-c2e88833-f3ef-4c3b-83c2-20f2956ec0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:32.538481 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e88833-f3ef-4c3b-83c2-20f2956ec0aa None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:32.543214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:32.543892 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:32.544248 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:32.544439 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] PATH_INFO: `/v3/projects/500e1fa77cb6492381d8b94dfda93b0f` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:32.545651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=500e1fa77cb6492381d8b94dfda93b0f)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:32.547547 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:32.660336 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '500e1fa77cb6492381d8b94dfda93b0f', 'request_id': 'req-c10386d6-1846-4781-b12e-965f0b693e69'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:32.734489 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '500e1fa77cb6492381d8b94dfda93b0f', 'request_id': 'req-c10386d6-1846-4781-b12e-965f0b693e69'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:32.740936 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '500e1fa77cb6492381d8b94dfda93b0f', 'request_id': 'req-c10386d6-1846-4781-b12e-965f0b693e69'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:32.752220 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] The token cache is being invalidate because project 500e1fa77cb6492381d8b94dfda93b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:40:32.752507 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:32.753137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-c10386d6-1846-4781-b12e-965f0b693e69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:32.754636 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1451/2920] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:40:32 2022] DELETE /identity/v3/projects/500e1fa77cb6492381d8b94dfda93b0f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:32.759620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc690058-9c2c-47b0-b23c-88630270c5c7 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:32.760129 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc690058-9c2c-47b0-b23c-88630270c5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:32.760232 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc690058-9c2c-47b0-b23c-88630270c5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:32.798480 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-fc690058-9c2c-47b0-b23c-88630270c5c7 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:32.799668 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-fc690058-9c2c-47b0-b23c-88630270c5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:32.858137 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1470/2921] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:32.867137 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35171389-c01d-4242-bd21-0118526fe95d None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:32.867419 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35171389-c01d-4242-bd21-0118526fe95d None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:32.867698 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35171389-c01d-4242-bd21-0118526fe95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:32.887883 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-35171389-c01d-4242-bd21-0118526fe95d None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:32.889076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-35171389-c01d-4242-bd21-0118526fe95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:32.896870 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1452/2922] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:32.901110 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc690058-9c2c-47b0-b23c-88630270c5c7 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:32.910746 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc690058-9c2c-47b0-b23c-88630270c5c7 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:32.915287 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569c9c3-9142-4798-9d0a-4a09c558715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:32.915917 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569c9c3-9142-4798-9d0a-4a09c558715f admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:32.916196 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569c9c3-9142-4798-9d0a-4a09c558715f admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:32.916471 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569c9c3-9142-4798-9d0a-4a09c558715f admin admin] PATH_INFO: `/v3/domains` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:32.917108 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569c9c3-9142-4798-9d0a-4a09c558715f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81775) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:40:32.917421 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569c9c3-9142-4798-9d0a-4a09c558715f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:32.919500 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569c9c3-9142-4798-9d0a-4a09c558715f admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:32.930739 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1471/2923] 10.222.0.15 () {58 vars in 1153 bytes} [Wed Sep 7 16:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:40:32.935458 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35171389-c01d-4242-bd21-0118526fe95d None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:32.944632 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35171389-c01d-4242-bd21-0118526fe95d None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:32.949554 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80bb92-eff3-4281-bf7c-7ceaaf7095ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:32.950139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80bb92-eff3-4281-bf7c-7ceaaf7095ce admin admin] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:32.950415 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80bb92-eff3-4281-bf7c-7ceaaf7095ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:32.950686 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c80bb92-eff3-4281-bf7c-7ceaaf7095ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:32.951319 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80bb92-eff3-4281-bf7c-7ceaaf7095ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:32.953214 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c80bb92-eff3-4281-bf7c-7ceaaf7095ce admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:33.044846 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1453/2924] 10.222.0.15 () {60 vars in 1153 bytes} [Wed Sep 7 16:40:32 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:40:33.049095 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569c9c3-9142-4798-9d0a-4a09c558715f admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:33.058992 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569c9c3-9142-4798-9d0a-4a09c558715f admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:33.113815 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1e00f-4560-4806-a18d-f1cdea6f63ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:33.114411 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1e00f-4560-4806-a18d-f1cdea6f63ec admin admin] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:33.114688 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1e00f-4560-4806-a18d-f1cdea6f63ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:33.114964 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1e00f-4560-4806-a18d-f1cdea6f63ec admin admin] PATH_INFO: `/v3/users` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:33.115577 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1e00f-4560-4806-a18d-f1cdea6f63ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:33.117504 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1e00f-4560-4806-a18d-f1cdea6f63ec admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:33.236464 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1472/2925] 10.222.0.15 () {60 vars in 1144 bytes} [Wed Sep 7 16:40:33 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:40:33.240611 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80bb92-eff3-4281-bf7c-7ceaaf7095ce admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:33.250710 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c80bb92-eff3-4281-bf7c-7ceaaf7095ce admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:33.255249 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235295a-d97a-4d08-9c72-2aa04f85e337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:33.255879 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235295a-d97a-4d08-9c72-2aa04f85e337 admin admin] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:33.256156 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235295a-d97a-4d08-9c72-2aa04f85e337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:33.256448 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235295a-d97a-4d08-9c72-2aa04f85e337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:33.257086 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235295a-d97a-4d08-9c72-2aa04f85e337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:33.258683 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235295a-d97a-4d08-9c72-2aa04f85e337 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:33.268444 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1454/2926] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:40:33.272243 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1e00f-4560-4806-a18d-f1cdea6f63ec admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:33.282766 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1e00f-4560-4806-a18d-f1cdea6f63ec admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:33.287392 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a2917-724b-4a5d-a1f1-c789b27971ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:33.288021 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a2917-724b-4a5d-a1f1-c789b27971ff admin admin] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:33.288306 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a2917-724b-4a5d-a1f1-c789b27971ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:33.288582 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a2917-724b-4a5d-a1f1-c789b27971ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:33.289218 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a2917-724b-4a5d-a1f1-c789b27971ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:33.290795 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5a2917-724b-4a5d-a1f1-c789b27971ff admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:33.301878 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1473/2927] 10.222.0.15 () {58 vars in 1122 bytes} [Wed Sep 7 16:40:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:40:33.305800 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235295a-d97a-4d08-9c72-2aa04f85e337 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:33.315577 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235295a-d97a-4d08-9c72-2aa04f85e337 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:33.320574 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd15fc-fa94-4746-b029-7634c1527876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:33.321200 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd15fc-fa94-4746-b029-7634c1527876 admin admin] REQUEST_METHOD: `PUT` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:33.321475 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd15fc-fa94-4746-b029-7634c1527876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:33.321746 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd15fc-fa94-4746-b029-7634c1527876 admin admin] PATH_INFO: `/v3/projects/59579596c615429c97753a12c817e8a4/users/8ca4919b944e4ae59f6dcaee94b1b2b6/roles/f463de99ba174eab9396eef514e34e15` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:33.335837 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd15fc-fa94-4746-b029-7634c1527876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59579596c615429c97753a12c817e8a4, user_id=8ca4919b944e4ae59f6dcaee94b1b2b6, role_id=f463de99ba174eab9396eef514e34e15)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:33.338440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd15fc-fa94-4746-b029-7634c1527876 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:33.414103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1455/2928] 10.222.0.15 () {60 vars in 1483 bytes} [Wed Sep 7 16:40:33 2022] PUT /identity/v3/projects/59579596c615429c97753a12c817e8a4/users/8ca4919b944e4ae59f6dcaee94b1b2b6/roles/f463de99ba174eab9396eef514e34e15 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:33.424419 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acff339-45a6-4ffe-848b-992e9f3a1533 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:33.437173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acff339-45a6-4ffe-848b-992e9f3a1533 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:33.437173 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acff339-45a6-4ffe-848b-992e9f3a1533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:33.444158 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:33.453300 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-0acff339-45a6-4ffe-848b-992e9f3a1533 None None] MFA Rules not processed for user `8ca4919b944e4ae59f6dcaee94b1b2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:33.454535 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-0acff339-45a6-4ffe-848b-992e9f3a1533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:33.516298 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1474/2929] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:33.526825 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd15fc-fa94-4746-b029-7634c1527876 admin admin] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:33.529162 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-dabd15fc-fa94-4746-b029-7634c1527876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:33.543474 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:33.585621 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd15fc-fa94-4746-b029-7634c1527876 admin admin] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:33.589509 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e23aa5-be33-497e-8246-4ada375a7750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f5f8dfbb9c44e788f110a4f53425344', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:33.590102 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23aa5-be33-497e-8246-4ada375a7750 service cinder] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:33.590968 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23aa5-be33-497e-8246-4ada375a7750 service cinder] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:33.591253 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e23aa5-be33-497e-8246-4ada375a7750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:33.599435 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e23aa5-be33-497e-8246-4ada375a7750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:33.600972 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e23aa5-be33-497e-8246-4ada375a7750 service cinder] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:33.610085 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1456/2930] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:35.382486 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3042c62-f7b2-4747-a827-3c946fabae85 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:35.383008 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3042c62-f7b2-4747-a827-3c946fabae85 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:35.383139 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3042c62-f7b2-4747-a827-3c946fabae85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:35.402569 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-c3042c62-f7b2-4747-a827-3c946fabae85 None None] MFA Rules not processed for user `8ca4919b944e4ae59f6dcaee94b1b2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:35.403833 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c3042c62-f7b2-4747-a827-3c946fabae85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:35.410621 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1475/2931] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:36.022497 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e23aa5-be33-497e-8246-4ada375a7750 service cinder] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:36.023997 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-d1e23aa5-be33-497e-8246-4ada375a7750 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:36.116469 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e23aa5-be33-497e-8246-4ada375a7750 service cinder] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:36.121663 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245d841-f456-49db-b44c-634ec24c817d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:36.122267 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245d841-f456-49db-b44c-634ec24c817d service placement] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:36.122549 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245d841-f456-49db-b44c-634ec24c817d service placement] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:36.122824 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245d841-f456-49db-b44c-634ec24c817d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:36.124652 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b245d841-f456-49db-b44c-634ec24c817d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:36.136275 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245d841-f456-49db-b44c-634ec24c817d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:36.137845 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b245d841-f456-49db-b44c-634ec24c817d service placement] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:36.147733 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1457/2932] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:45.397205 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=31 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:46.144184 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=61 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:47.186750 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3042c62-f7b2-4747-a827-3c946fabae85 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:47.187210 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-c3042c62-f7b2-4747-a827-3c946fabae85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:47.291597 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3042c62-f7b2-4747-a827-3c946fabae85 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:47.297167 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acdc97-d409-4249-a7d4-dcb6b6a1496c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45061db68b1e40f59935cc9a46d2ff10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:47.297794 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acdc97-d409-4249-a7d4-dcb6b6a1496c service nova] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:47.298070 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acdc97-d409-4249-a7d4-dcb6b6a1496c service nova] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:47.298370 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acdc97-d409-4249-a7d4-dcb6b6a1496c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:47.300073 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-e0acdc97-d409-4249-a7d4-dcb6b6a1496c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:47.312829 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acdc97-d409-4249-a7d4-dcb6b6a1496c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:47.314394 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acdc97-d409-4249-a7d4-dcb6b6a1496c service nova] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:47.325652 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1476/2933] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:52.926718 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245d841-f456-49db-b44c-634ec24c817d service placement] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:52.928157 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-b245d841-f456-49db-b44c-634ec24c817d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:52.941651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245d841-f456-49db-b44c-634ec24c817d service placement] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:52.946850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e398a-9944-4471-951b-707316b015e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5fdfb0bf09db406ab5c09fbef887cbab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:52.947440 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e398a-9944-4471-951b-707316b015e1 service placement] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:52.947719 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e398a-9944-4471-951b-707316b015e1 service placement] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:52.948021 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e398a-9944-4471-951b-707316b015e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:52.949669 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-709e398a-9944-4471-951b-707316b015e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:52.960723 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e398a-9944-4471-951b-707316b015e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:52.962269 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e398a-9944-4471-951b-707316b015e1 service placement] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:52.972115 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1458/2934] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:54.081074 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bc4a4-8b56-4c95-b2bf-0589d08377a9 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:54.087348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bc4a4-8b56-4c95-b2bf-0589d08377a9 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:54.087348 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bc4a4-8b56-4c95-b2bf-0589d08377a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:54.111239 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-be4bc4a4-8b56-4c95-b2bf-0589d08377a9 None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:54.112556 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-be4bc4a4-8b56-4c95-b2bf-0589d08377a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:54.170040 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1477/2935] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:40:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:54.307211 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e398a-9944-4471-951b-707316b015e1 service placement] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:54.309233 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-709e398a-9944-4471-951b-707316b015e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:54.377401 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e398a-9944-4471-951b-707316b015e1 service placement] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:54.382395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3652dba-68c5-4c78-9cb4-d70fea94458c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:54.382988 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3652dba-68c5-4c78-9cb4-d70fea94458c service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:54.383268 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3652dba-68c5-4c78-9cb4-d70fea94458c service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:54.383568 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3652dba-68c5-4c78-9cb4-d70fea94458c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:54.392100 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3652dba-68c5-4c78-9cb4-d70fea94458c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:54.393670 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3652dba-68c5-4c78-9cb4-d70fea94458c service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:54.401613 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1459/2936] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:40:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:40:55.197405 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d82b55-a51c-4e45-ae52-c6d5b67cc9b3 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:55.197880 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d82b55-a51c-4e45-ae52-c6d5b67cc9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:55.197999 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d82b55-a51c-4e45-ae52-c6d5b67cc9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:55.219466 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-57d82b55-a51c-4e45-ae52-c6d5b67cc9b3 None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:55.220792 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-57d82b55-a51c-4e45-ae52-c6d5b67cc9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:55.227840 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1478/2937] 10.222.0.15 () {58 vars in 967 bytes} [Wed Sep 7 16:40:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:55.240188 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3652dba-68c5-4c78-9cb4-d70fea94458c service swift] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:55.250754 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3652dba-68c5-4c78-9cb4-d70fea94458c service swift] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:55.255755 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e181937-ab0e-4a9d-80aa-970c09951f83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:55.256374 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e181937-ab0e-4a9d-80aa-970c09951f83 service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:55.256651 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e181937-ab0e-4a9d-80aa-970c09951f83 service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:55.256925 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e181937-ab0e-4a9d-80aa-970c09951f83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:55.266571 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e181937-ab0e-4a9d-80aa-970c09951f83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:55.268173 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e181937-ab0e-4a9d-80aa-970c09951f83 service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:55.276076 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1460/2938] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:40:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:40:55.835098 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230a6c-e5ba-45f0-aeec-087088c4f1a1 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:55.835380 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230a6c-e5ba-45f0-aeec-087088c4f1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:55.835657 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230a6c-e5ba-45f0-aeec-087088c4f1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:55.854358 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-bd230a6c-e5ba-45f0-aeec-087088c4f1a1 None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:55.855554 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-bd230a6c-e5ba-45f0-aeec-087088c4f1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:55.862608 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1479/2939] 10.222.0.15 () {58 vars in 1009 bytes} [Wed Sep 7 16:40:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:55.874858 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e181937-ab0e-4a9d-80aa-970c09951f83 service swift] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:55.885476 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e181937-ab0e-4a9d-80aa-970c09951f83 service swift] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:55.891252 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0604-94f9-4930-b353-a78f078c1ac7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:55.891867 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0604-94f9-4930-b353-a78f078c1ac7 service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:55.892239 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0604-94f9-4930-b353-a78f078c1ac7 service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:55.892515 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d0604-94f9-4930-b353-a78f078c1ac7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:55.900429 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d0604-94f9-4930-b353-a78f078c1ac7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:55.901981 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d0604-94f9-4930-b353-a78f078c1ac7 service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:55.909884 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1461/2940] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:40:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:40:56.509198 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53052c0-34bb-44bf-9dd7-2ac709fde816 None None] REQUEST_METHOD: `POST` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:56.509667 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53052c0-34bb-44bf-9dd7-2ac709fde816 None None] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:56.509785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53052c0-34bb-44bf-9dd7-2ac709fde816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:56.529999 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.auth.core [None req-b53052c0-34bb-44bf-9dd7-2ac709fde816 None None] MFA Rules not processed for user `6b6893f49c96479c8f8dd11a372c257c`. Rule list: `[]` (Enabled: `True`). {{(pid=81775) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:56.531223 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-b53052c0-34bb-44bf-9dd7-2ac709fde816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:56.542374 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1480/2941] 10.222.0.15 () {58 vars in 967 bytes} [Wed Sep 7 16:40:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:56.555992 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0604-94f9-4930-b353-a78f078c1ac7 service swift] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:56.566648 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d0604-94f9-4930-b353-a78f078c1ac7 service swift] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:56.571994 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f69a5e-150d-4d0d-a80e-46f744530b9e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69a8796195ff4dfb85655cdd91a0229d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:56.572584 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f69a5e-150d-4d0d-a80e-46f744530b9e service swift] REQUEST_METHOD: `GET` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:56.572863 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f69a5e-150d-4d0d-a80e-46f744530b9e service swift] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:56.573139 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f69a5e-150d-4d0d-a80e-46f744530b9e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:56.582494 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f69a5e-150d-4d0d-a80e-46f744530b9e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:56.584103 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f69a5e-150d-4d0d-a80e-46f744530b9e service swift] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:56.592727 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1462/2942] 10.222.0.15 () {60 vars in 1384 bytes} [Wed Sep 7 16:40:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:40:59.229854 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53052c0-34bb-44bf-9dd7-2ac709fde816 None None] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:59.268180 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=100 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:40:59.291242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53052c0-34bb-44bf-9dd7-2ac709fde816 None None] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:59.295782 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:59.296420 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:59.296700 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:59.296974 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] PATH_INFO: `/v3/users/8ca4919b944e4ae59f6dcaee94b1b2b6` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:59.298120 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca4919b944e4ae59f6dcaee94b1b2b6)` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:59.300067 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:59.466689 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca4919b944e4ae59f6dcaee94b1b2b6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:59.543588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca4919b944e4ae59f6dcaee94b1b2b6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:59.544136 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca4919b944e4ae59f6dcaee94b1b2b6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:59.544731 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca4919b944e4ae59f6dcaee94b1b2b6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:59.551620 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.notifications [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca4919b944e4ae59f6dcaee94b1b2b6'} {{(pid=81775) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:40:59.568983 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1481/2943] 10.222.0.15 () {58 vars in 1224 bytes} [Wed Sep 7 16:40:59 2022] DELETE /identity/v3/users/8ca4919b944e4ae59f6dcaee94b1b2b6 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:40:59.573752 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb0008-efc2-4816-9846-936abb725654 None None] REQUEST_METHOD: `POST` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:59.574242 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb0008-efc2-4816-9846-936abb725654 None None] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:59.574357 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfb0008-efc2-4816-9846-936abb725654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:59.593714 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.auth.core [None req-7dfb0008-efc2-4816-9846-936abb725654 None None] MFA Rules not processed for user `f2181a60abf646d28f09241ba111a2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81776) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:40:59.594917 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7dfb0008-efc2-4816-9846-936abb725654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:59.651711 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1463/2944] 10.222.0.15 () {58 vars in 958 bytes} [Wed Sep 7 16:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:40:59.661215 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] Authenticating user token {{(pid=81775) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:59.662588 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.fernet_utils [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81775) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:59.718785 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af94a4-84c7-42c8-90b5-c7dcb131a563 admin admin] Validating token access rules against request {{(pid=81775) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:59.722381 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f55969-fc74-44ca-ba07-c88f0d3fa217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a6b52462624e7f84f479144353b51e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c479846c3d274061a55c19a95ddb5044', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81775) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:59.722967 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f55969-fc74-44ca-ba07-c88f0d3fa217 service neutron] REQUEST_METHOD: `GET` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:59.723242 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f55969-fc74-44ca-ba07-c88f0d3fa217 service neutron] SCRIPT_NAME: `/identity` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:59.723510 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f55969-fc74-44ca-ba07-c88f0d3fa217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81775) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:59.732189 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f55969-fc74-44ca-ba07-c88f0d3fa217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:40:59.734162 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f55969-fc74-44ca-ba07-c88f0d3fa217 service neutron] RBAC: Authorization granted {{(pid=81775) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:40:59.743340 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: [pid: 81775|app: 0|req: 1482/2945] 10.222.0.15 () {60 vars in 1365 bytes} [Wed Sep 7 16:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:40:59.975361 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb0008-efc2-4816-9846-936abb725654 None None] Authenticating user token {{(pid=81776) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:40:59.976852 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.fernet_utils [None req-7dfb0008-efc2-4816-9846-936abb725654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81776) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:40:59.990533 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfb0008-efc2-4816-9846-936abb725654 None None] Validating token access rules against request {{(pid=81776) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:40:59.995395 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f2181a60abf646d28f09241ba111a2d3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f97db8bc2c2455d85720c36a1ed180d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81776) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:40:59.996065 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:40:59.996344 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:40:59.996717 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] PATH_INFO: `/v3/projects/59579596c615429c97753a12c817e8a4` {{(pid=81776) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:40:59.998417 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59579596c615429c97753a12c817e8a4)` {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:41:00.000377 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] RBAC: Authorization granted {{(pid=81776) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:41:00.092206 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:41:00.103914 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59579596c615429c97753a12c817e8a4', 'request_id': 'req-40a8e1bc-4758-41eb-862b-0550911ed6d8'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:41:00.196042 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59579596c615429c97753a12c817e8a4', 'request_id': 'req-40a8e1bc-4758-41eb-862b-0550911ed6d8'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:41:00.202848 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59579596c615429c97753a12c817e8a4', 'request_id': 'req-40a8e1bc-4758-41eb-862b-0550911ed6d8'} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:41:00.214688 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] The token cache is being invalidate because project 59579596c615429c97753a12c817e8a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81776) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:41:00.214980 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:41:00.215659 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG keystone.notifications [None req-40a8e1bc-4758-41eb-862b-0550911ed6d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81776) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:41:00.217813 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: [pid: 81776|app: 0|req: 1464/2946] 10.222.0.15 () {58 vars in 1233 bytes} [Wed Sep 7 16:40:59 2022] DELETE /identity/v3/projects/59579596c615429c97753a12c817e8a4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:41:09.739311 dv-cinder-856174-1-iscsi devstack@keystone.service[81775]: DEBUG dbcounter [-] [81775] Writing DB stats keystone:SELECT=63,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81775) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:41:10.276850 dv-cinder-856174-1-iscsi devstack@keystone.service[81776]: DEBUG dbcounter [-] [81776] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81776) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}